Show HN: Ex Situ – Open-source spatial index of displaced cultural artifacts

46 points by hbyel 11 hours ago

Hi, I designed and developed a spatial index that maps museum artefacts as connecting arcs/hyperlinks from their origin site to institutional location/sources. The Index specifically looks at western/euro-american institutions and maps their collection categorised by them under Islamic art, Asian/African art, ethnological collections, Middle East, South America etc.

Started as my MA thesis in 2022, kept building since, mostly solo with a little funding. Fully open source, self-hosted, AGPL-3.0, Next.js + Deck.gl on the frontend, Strapi backend, Python ETL pulling from museum open-access APIs. Currently indexing over 100k artifacts from all over the world, across 8 collections (Met, V&A, SMB Berlin). Recently added an md export so researchers can download provenance data for a filtered set of artifacts.

The infrastructure is conceptualized as an indexer, not a hoster. Even images are kept as URLs pointing to the source institutions. It's connective tissue between archives that were never designed to speak to each other, with an origin-first search UI concept. The data model is on purpose flat, to avoid encoding problematic taxonomies, and routes researchers directly to the source giving responsibility to the source institutions rather than duplicating institutional data. It indexes the relationship between origin sites and destination collections.

Coming from a design background, taking this project from prototype to production app has been incredibly rewarding, but honestly it’s getting a little bit overwhelming to scale alone. I would love community’s feedback on performance scaling, any code contributions, data pipelines for missing museum apis or general feedback.

Repo: https://github.com/hburakyel/ex-situ

Live: https://exsitu.app

theroadnotbacon - 9 hours ago

Huh! Was hoping to get, for example, a sense of scale of cultural artifacts stolen from Korea by the Japanese. But I don't see any provenance descriptions for Japanese institutions. Either way, this is super cool. Thank you for sharing this.

hbyel - 8 hours ago

Fair catch, it only pulls from institutions with open access APIs (Met, V&A, SMB Berlin, AIC). Not aware of an equivalent from Japanese museums, if you know one, happy to add it.
- leobg - 7 hours ago
  
  In that case, the map looks misleading, doesn’t it? It looks as though London and Berlin, for example, stand out as having dislocated disproportionately many artifacts from around the world. And on the other hand, Paris looks innocent. But in truth, London and Berlin are just being honest about it giving you open access to the data whereas Paris doesn’t. So perhaps it would be better to mark cities that do provide data with, say, a green dot. This also helps the visual understanding of the direction of the transfer. And it would make it clear at a glance that Paris is not innocent, but that Paris is simply lacking data.
  - andrewaylett - 13 minutes ago
    
    I was slightly discombobulated by the first London link I clicked actually showing a number of artifacts from London that are currently held in Berlin.
  - hbyel - 7 hours ago
    
    Fair point, and a good idea. A visual marker distinguishing "has open data" from "doesn't" would meaningfully reduce that distortion. Adding it to the list.
  - NoboruWataya - 5 hours ago
    
    > London and Berlin are just being honest about it giving you open access to the data
    A small subset of the data. If all London museums were included (such as the British Museum, for example) there would be a LOT more lines going to London.

txzl - 8 hours ago

I think much more lines should go to Moscow. At least from such places as Crimea. But I see you covered only western/euro-american institutions.

hbyel - 8 hours ago

Correct, scoped to western/euro-american museums as noted in the post, it’s just where I started. Open to expanding if there’s good open data to pull from.

satoyoshidev - an hour ago

The README says arc aggregation runs in a web worker. At 200k artifacts, what actually chokes first when everything is visible at low zoom?

iamacyborg - 9 hours ago

One day I really should dig up my great great great great grandfather’s[^1] diaries to see first hand how he justified bringing back so much stuff to the UK.

[^1]: https://en.wikipedia.org/wiki/William_Richard_Hamilton

fkdk - 8 hours ago

“To arrive at a just estimate of a renowned man’s character one must judge it by the standards of his time, not ours.”
- mad_tortoise - 5 hours ago
  
  Crap quotes like this is what people use to justify slavery, fascism, genocide, apartheid.
  - kakacik - 2 hours ago
    
    Does the world really need to be reduced to binary state, people being black or white and nothing else? I know its supremely convenient mentally, this is how populists ride their huge waves and its each of us in the crowd that allow that.
    I find it much more compelling to fully see some negative aspect(s) of someone's past existence while accepting that if I grew up in that era, I would have vastly different morals and probably be fine with that.
    Judge others not too harshly if you may, since you will also be judged further down the line and most surely will be judged as inadequately moral or good by their standards, didn't do enough to help preventing some proper shit happening/will happen and so on, rather looked away while milking some comfy job for those sweet dollars (we all have kids and families and mortgages and dreams I know, so does everybody else).
    Most people here work for FAANGs of this era for example, hardly moral corporations by any metric and some literally sowing evil ie in cancers called social networks and their gamifications or pushing unwanted ads / outright lies into every eyeball they can reach. And those companies are hardly the only black spots on current society, I'd even say purely moral work is rather rare.
    
    mad_tortoise - 2 hours ago
    
    I judge people exactly how my moral's and principles determine they should be judged. People love to use "of the time" type quotes to excuse slavery for example, when in actual fact it was well known at the time to be despicable.
    Or perhaps a better example is "of the time" quotes to justify apartheid South Africa, a place my parents grew up in. Many friends parent's were blindly ignorant or supportive, but that doesn't mean they didn't know the facts. They just go, "that's how it was". When I know from my parent's it wasn't like that, and White and Black people knew exactly what was going on. Same could be said for Nazi Germany.
    And certainly same can be said for everyone on here that works at FAANG, or Palantir or any US defense contractor. They are knowingly and willingly aiding genocide or mass surveillance. They cannot claim ignorance.
    We live in a day and age where people far too easily give up their freedoms, give up moral stand points because it is difficult or divisive. Yes, that's exactly what it is meant to be.
    Maybe if we judged people more harshly we wouldn't be seeing a rise in fascism today, because views are just accepted as they are. Hell no. I hope I am judged down the line. Don't be too blase about accepting shitty viewpoints because people are morally ignorant, or lack principles and in doing so divide humanity along spurious lines.
leobg - 7 hours ago

You know that they exist, you know where they are, and you’ve never read or digitized them? Or was that sarcasm?
- iamacyborg - 6 hours ago
  
  They’re at my aunts house, I’ve just not had a chance to read them or ask my cousin to digitise them and send them to me.
  - leobg - 3 hours ago
    
    Looks like you've got some real history there. Think about it. You might even be able to hack together an Ask My Ancestor chatbot. Heck, I’d build it for you. The scamming and the OCR are going to be the hardest parts I imagine.
    
    iamacyborg - 3 hours ago
    
    I’m quite fortunate on the family history side of things. Not sure a chatbot is what’s needed but I’d be happy to make digitised files public that relate to things that are in the public interest. This ancestor for example was key in the acquisition of the rosetta stone and the parthenon marbles and was the recipient of one of Canova’s “Ideal Heads” that can now be seen at the Ashmolean in Cambridge. There’s a lot of history associated with those 3 events.

arendtio - 7 hours ago

> Coming from a design background

From a UX perspective, it is very weird to place the toggle button for the right panel at the top of the left panel... After I gave up to find a way to close the right panel, I found that button a few minutes later, when I looked around for other interactive elements.

hbyel - 7 hours ago

thanks, agreed, will fix :)

FelixKunzJr - 3 hours ago

Is there any way to filter the lines? There are so many lines it's hard to see what's going on. But definitely interesting.

hbyel - 3 hours ago

Thanks, you can filter by source in the search panel already.

poulpy123 - an hour ago

What counts as displaced ?

potamic - 8 hours ago

This is fantastic visualization!

I couldn't figure out from the UI but is there a way to filter the map? Say, I only want to see things going out of Cusco or I only want to see where things end up in Berlin?

It would also be nice to have some directionality to the lines to indicate source and destination, perhaps a colour or lightness gradient or something.

hbyel - 7 hours ago

Thanks! You actually can filter by source already, search lets you filter arcs by place/site. Destination filtering technically works too, via Collections, but that only makes sense while there are a handful, once it’s hundreds it’ll need its own proper destination filter. Good call on the color/gradient distinction for source vs. destination on the arcs too, that’s worth trying.
- ahk-dev - 7 hours ago
  
  One thing I'd find interesting is the inverse view: instead of starting from a place or artifact, start from a museum and visualize the geographic footprint of its collection. It could make the collection's provenance surprisingly tangible, especially for institutions with very global holdings.

vanyaland - 6 hours ago

How well does the fuzzy matcher resolve provenance text to origin sites? Curious what share of the 200k in the README needed hand correction.

hbyel - 5 hours ago

[flagged]

bazzingadev - 7 hours ago

Do you plan on improving the performance of the website ?

hbyel - 7 hours ago

[flagged]

warpspeedchic - 8 hours ago

holy gosh, this is brilliant. i've been toggling around this site for a good 40 minutes now. learning so much!!! arrrgggh, i love the internet.

berardino - 8 hours ago

It is truly very useful not only for art history but also for the history of colonial rule

robintiman11 - 8 hours ago

Very cool

throwaway69123 - 2 hours ago

now make a database of cultural artifacts destroyed by origin countries

terekhindc - 6 hours ago

[dead]