Towards Accountability in Machine Learning Applications: A System-testing Approach

New working paper, with Wayne Xinwei Wan

A rapidly expanding universe of technology-focused startups is trying to change and improve the way real estate markets operate. The undisputed predictive power of machine learning (ML) models often plays a crucial role in the ‘disruption’ of traditional processes. However, an accountability gap prevails: How do the models arrive at their predictions? Do they do what we hope they do – or are corners cut?

Training ML models is a software development process at heart. We suggest following the dedicated software testing framework and verifying that the ML model is performing as intended. Illustratively, we augment two image classifiers with a system testing procedure based on local interpretable model-agnostic explanation (LIME) techniques. Analyzing the classifications sheds light on some of the factors that determine the behavior of the systems. We show that cross-validation is simply not good enough when operating in regulated environments.

Full paper: Towards Accountability in Machine Learning Applications: A System-testing Approach

Areas that are most important for automatic vintage classification (Example)

The Total Return and Risk to Residential Real Estate

New paper with Piet Eichholtz, Matthijs Korevaar, and Ronan Tallec;
accepted for publication at Review of Financial Studies.


We estimate total returns to rental housing by studying over 170,000 hand-collected archival observations of prices and rents for individual houses in Paris (1809–1943) and Amsterdam (1900–1979). The annualized real total return, net of costs and taxes, is 4.0% for Paris and 4.8% for Amsterdam, and entirely comes from rental yields. Our returns correlate weakly with the implied returns in Jorda et al. (2019) and are substantially lower. We decompose total return risk at the individual asset level, and find that yield risk becomes an increasingly important component of property-level risk for longer investment horizons.

Real Total Returns on Residential Real Estate, Paris
Real Total Returns on Residential Real Estate, Amsterdam

Full paper: The Total Return and Risk to Residential Real Estate

Are Estimates of Rapid Growth in Urban Land Values an Artifact of the Land Residual Model?

New publication, with John Clapp and Jeff Cohen, accepted for publication at the Journal of Real Estate Finance and Economics.

Separating urban land and structure values is important for national accounts and for analysis of house price dynamics. A large part of the literature on urban land valuation uses the land residual method, which relies on the assumption that structures are easily replaced. But urban land value depends on accessibility to nearby land uses, implying that infrastructure and the slowly changing built environment are the most important components of land value. Investments in structures are only slowly reversible, implying that land and structure function as a bundled good whereas land residual theory severs the connection between land value and structure value over time. We develop a simple theoretical model that includes risk – and therefore the option to delay – and compare our model to a nested land residual model before and after a shock to values. Cross-sectionally our model shows that land residual theory overestimates structure value; over time almost all of any change in property value is allocated to land residuals. Data from Maricopa county, AZ, 2012–2018, strongly support option value models when nested within a general model that also includes land residuals. FHFA estimates use entirely different cost estimation methods: our analysis of FHA data suggest that our conclusions generalize to the U.S. as a whole, and that high and rising land value ratios (the “hockey stick” pattern found in the literature) are likely an artifact of the residual model. We further show that construction costs are valued by the housing market, suggesting a new use of the construction cost variable.

Full paper: Are Estimates of Rapid Growth in Urban Land Values an Artifact of the Land Residual Model?

Machine Learning, Architectural Styles and Property Values

New paper with Erik B. Johnson, accepted for publication at the Journal of Real Estate Finance and Economics.

This paper couples a traditional hedonic model with architectural style classifications from human experts and machine learning (ML) enabled classifiers to estimate sales price premia over architectural styles, both at the building and the neighborhood-level. We find statistically and economically significant price differences for houses from distinct architectural styles across an array of specifications and modeling assumptions. Comparisons between classifications from ML models and human experts illustrate the conditions under which ML classifiers may perform at least as reliable as human experts in mass appraisal models. Hedonic estimates illustrate that the impact of architectural style on price is attenuated by properties with less well-defined styles and we find no evidence for differential price effects of Revival or Contemporary architecture for new construction.

Automatic Vintage Classification, using Street-level Imagery

Link to Paper: Machine Learning, Architectural Styles and Property Values

Real Estate Finance and Investment Symposium, Sep 25-27, 2019

University of Cambridge
University of Florida
National University of Singapore

Real Estate Finance and Investment Symposium
September 25-27, 2019
Call for Papers

The University of Cambridge, the University of Florida, and the National University of Singapore (NUS) announce a call for papers for their joint 2019 Real Estate Finance and Investment Symposium. The symposium will feature longer, more in-depth paper presentations and will allow ample time for discussion among presenters, assigned discussants, and other participants. The event will be held in a relaxed yet focussed setting aiming to inspire a critical mass of leading academics in the field to create new thoughts and insights on general finance, real estate finance and investments topics.

The symposium will be hosted by the Cambridge Real Estate Research Centre and will take place at St John’s College, Cambridge. Accommodation (College guest rooms, en suite) and group meals will be provided by the organising institutions. All travel costs will be borne by the participants.
The organizers do not wish to necessarily limit the focus of the conference, and papers on any real estate-related finance or economics topic are welcome. However, subject to sufficient interest, we may try to focus the symposium on one or more of the following themes:

  • Real estate risk management
  • Leverage, capital structure and real estate
  • Infrastructure investment and privatization
  • Real estate price index methodology and performance measurement
  • Economics of sustainability in real estate
  • International real estate investment
  • “Big Data,” ML and AI in real estate finance research
  • Behavioural approaches to real estate
  • Derivatives

Paper Submission

The deadline for submission of papers is June 1, 2019. From the submissions, the organizers will select approximately eight papers for presentation at the symposium, with notification to authors by July 1st, 2019. All accepted authors may choose to submit their papers for publication in a special issue of the Journal of Real Estate Finance and Economics with an expedited review subsequent to the symposium. Submission of full papers is highly encouraged.

Authors should submit their papers electronically to:
Thies Lindenthal (, Cambridge Real Estate Research Centre

The Program

The participants are expected to arrive on Wednesday, September 25. All presentations will be on Thursday and Friday (September 26-27). There will be no parallel sessions, and all participants are expected to attend all sessions, and play an active role in them. Each paper presentation session will last at least 45-50 minutes, including sufficient time for paper presentation, formal discussion, and general Q&A with the audience.


  • Wednesday, September 25
    • Pre-conference reception and dinner
  • Thursday, September 26
    • 9:00 AM, 3 paper sessions
    • 12:00 PM, Lunch
    • 2:00 PM, 3 paper sessions
    • 6:00 PM, Conference Dinner
  • Friday, September 27
    • 9:00 AM, 3 paper sessions
    • 12:00 PM, Lunch
    • 2:00 PM, conference closing


  • University of Florida: Wayne Archer, David Ling, Andy Naranjo
  • National University of Singapore: Joseph Ooi and Tien Foo Sing
  • University of Cambridge: Colin Lizieri and Thies Lindenthal

Advisory Board

  • Brent Ambrose, PennState
  • Piet Eichholtz, Maastricht University
  • Roland Fuess, University of St. Gallen
  • David Geltner, MIT
  • Martin Hoesli, University of Geneva
  • Rose Lai, University of Macau
  • Crocker Liu, Cornell University
  • Gianluca Marcato, University of Reading
  • Tim Riddiough, University of Wisconsin Madison
  • C.F. Sirmans, Florida State University
  • Sheridan Titman, University of Texas at Austin
  • Yildiray Yildirim, Baruch College, NY

Estimating Supply Elasticities for Residential Real Estate in the United Kingdom

Elsevier’s “Reference Module in Earth Systems and Environmental Sciences” features a new chapter of mine on Digital Elevation Models (DEM) and how to utilise them in economic research.

A rare example of consensus between residents, urban planners, property developers, investors, policy makers, and academics in the United Kingdom is the perception of the nation’s housing markets being under considerable stress: Britain’s home-builders are providing too few residential units to meet the increased demand for housing. Population growth, demographic changes, changes in tenancy preferences, and overall lower interest rates led to more and more households chasing after an only slowly expanding stock of properties, driving prices up and affordability down to new record levels—at least in the island’s south and south-east. Put differently, increases in demand for housing do not induce the construction of sufficiently many dwellings. The supply of UK housing appears to be inelastic…



Land Assembly in Amsterdam, 1832-2015

Together with Piet Eichholtz and David Geltner, I have written a new paper on land assembly and urban renewal spanning 180 years. It has just been accepted for publication in Regional Science and Urban Economics.


Inner city redevelopment frequently involves the assembly of small lots into bigger ones. We analyze joint lot development and the influence of coordination and transaction costs of land assembly on the exercise of the redevelopment option, using Amsterdam micro housing information for 1832, 1860 and 2015. In all, we have a complete set of building structure and household characteristics for dwellings on almost 30,000 lots for each of these years. We estimate a logit model to predict joint lot redevelopment, based on structural characteristics of lots and dwellings and on social characteristics of their occupants.The results show that both types of characteristics significantly explain land assembly, and the regression coefficients adhere to the theoretical land assembly literature. This paper contributes importantly to our knowledge of the specific land parcel and structural physical characteristics that impact redevelopment. To our knowledge, this is the first paper to study the joint characteristics of the potentially combinable lots, and to document and quantify the role of social characteristics in land assembly.

Here the full manuscript

Notes: The maps above provide information on the pairwise redevelopment of lots between 1832 and 1860, and between 1832 and 2015. Redeveloped lots are denoted in red, unchanged lots in blue. The maps are based on Amsterdam’s cadastral maps for 1832, 1860, and 2015.

Long-run House Prices in Norway, 1819-2015

Using the fantastic indices estimated by Eitrheim and Erlandsen (2004), Arne Eichholtz visualised the long term price dynamics (or rather the lack thereof in most years) for Norwegian cities.


  • Eitrheim, Ø. and S. Erlandsen (2004). “House price indices for Norway 1819–2003”, in: Eitrheim, Ø., Klovland, J.T., Qvigstad, J.F. (Eds.), Historical Monetary Statistics for Norway. Norges Bank, Oslo, pp. 349–375.
  • Grytten, O.H., 2004. A Consumer Price Index for Norway 1516-2003, in: Eitrheim, Ø., Klovland, J.T., Qvigstad, J.F. (Eds.), Historical Monetary Statistics for Norway. Norges Bank, Oslo, pp. 518–519.
  • (2016). Consumer Price Index
  • (2016). House Price Indices



A different kind of job market

The new project my brother Wieland is heavily involved in is making great strides towards establishing a new market for small jobs in Spain: jobMapp aims to connect jobbers with potential customers. While I am not sure I would hire somebody to to my household chores, I am very excited to see that one can now arrange deliveries of documents and small parcels in Madrid through a dedicated website:

What will be next? Pool cleaning? Redecorating? Moving services? Maid and cleaning services?

Monocentric Cyberspace: The Primary Market for Internet Domain Names

One of my research paper has been accepted for publication in the
Journal of Real Estate Finance and Economics:


Cyberspace is no different from traditional cities, at least in economic terms. Urban economics governs the creation of ew space on the Internet and explains location choices and price gradients in virtual space. This study explores registration dynamics in the largest primary market for virtual space: Internet domain names. After developing a framework for domain registrations, it empirically tests whether domain registrations are constrained by the depletion of unregistered high quality domain names. Estimations based on registrations of COM domain names suggest that the number of domains expands substantially slower than the growth in overall demand for domain space. Supplying alternative domain extensions can relax the shortage in domains in the short term.

Read full paper: Monocentric Cyberspace (PDF)