Otodom Scraper: city and province were empty on every row

What happened

Otodom.pl moved the city and province of a listing out of the address block its search pages carry and into a separate geocoded location list. Our mapper read the address block, so from the day of the change every row the scraper delivered had city and province empty, with the rest of the row intact.

The change surfaced first on the sibling actor, Otodom Price Stats, which failed on 6 October because it groups listings by city. The fix was applied there the same day and not carried to this scraper until 9 October.

Impact

Every run between the site's change and 9 October delivered rows without city and province, charged as normal rows. Price, area, rooms, price per square metre and the listing address were correct. We do not see our customers' runs; our own scheduled runs on the sibling actor show the first affected day as 5 October. If you ran the scraper in that window and those two fields matter to you, open an issue on the actor's Issues tab with the run, and we will look at the charge with you.

Root cause

A change on the site's side, and a fix made to one actor of a family and not carried to its sibling the same day. Our checks counted rows and did not notice two fields going empty on all of them.

Fix

Shipped 9 October (build 0.1.17). City and province are read from the geocoded list, matched by level so a district or a street never stands in for a city, with the old address field kept as a fallback. A fixture captured from the live page pins the new shape.

What changed so it cannot recur this way

A change found on one actor of a family is now carried to its siblings in the same session, which is how the 9 October fix went out; and the live-shape fixture fails the build if the field goes empty again.

Questions about this incident? contact@lowlanddata.com - reports get read the same day.