Well, as promised, my travel calendar finally calmed down after May and I got to start shipping again 😊
surprise! Shipped something on a Friday 🙂 endpoints.bsky.app is the new home for the HTTP API reference that used to live in docs.bsky.app — should be much easier to reference for quick lookups and you can even mock up / test endpoint responses in the browser!
I got a really nice reception to my managing to push this out the door on a Friday, and among the responses were some requests for a write-up on how I did it, which was a great suggestion, thank you! Let me go into a bit of detail on why/what we tackled here:
Previously, we had an HTTP endpoint reference that lived on docs.bsky.app. This predated me, and when I joined and set to work on docs, one of my first questions (after "why do we have two docs sites," more on that in a bit) was "why do the Bluesky docs take 20 minutes to build?"
Well, as you might start to wonder from the +6k/-80k line PR that I pushed up to the Bluesky docs repo when I removed the old HTTP reference... it turns out that embedding this huge set of reference pages in Docusaurus was not particularly efficient! It wasn't all that nice to browse in the docs sidebar either, for that matter.
The reason we needed this previously was because: a) we'd been getting docs.bsky.app launched at an earlier stage of the project, b) Docusaurus is a very good framework if you're understandably hesitant to split your maintenance responsibility and your audience comprehension against multiple different static sites, c) our Lexicon-native codegen wasn't yet as mature and so we needed a usable way to map XRPC methods to APIs that new developers could grasp, and d) we wanted to expose all of this using OpenAPI. These are, imo, a super reasonable set of requirements that balance a lot of typical tensions in docs — we can't expect everyone to care about our internal tooling until we've made the utility of what we're doing extra clear, and we have to solve for clarity first.
Also, in general, docs.bsky.app hadn't gotten a lot of love from me or from anybody else in the last ~9 months; it'd basically been in maintenance mode (a situation not helped by the maintenance builds taking 20 minutes) while we modernized and shifted a lot of sensemaking over to atproto.com. Now, we finally have some new and exciting plans for the dedicated Bluesky docs which I'll be able to share soon, and as we were kicking off that work, we got to ask ourselves which parts of this we wanted to greenfield.
It was a bit unintuitive (I think I only got buy-in for this from my colleagues because I prototyped it really quickly), but I concluded that the part I wanted to greenfield with a brand new site was also the technical debt fix — the new HTTP reference. Carving that out of Docusaurus meant that we could then gradually update what is now a much more maintainable Docusaurus site, ship this in the meantime, and keep the net-new parts to neatly scoped improvements which would be appreciated by both our new and existing dev community.
This also made the requirement to scope a new OpenAPI docs implementation a lot more straightforward — Scalar was the obvious choice. It does this one thing really well, it's open source and as maintainable as any other MDX platform (we build and deploy our static sites with Github CI -> Render, which is working well for us), and we have no real need to migrate other content away from docs.bsky.app or atproto.com right now, so it didn't present many more complex questions.
I also wanted to go as codegen-forward as possible, and I'm very proud to say that all the descriptive text in these docs other than the Intro paragraphs comes directly from the published Lexicons themselves. What we've shipped here is, essentially, a Lexicon-to-OpenAPI human-readable renderer, which also maintains all the existing machine-readable API surfaces that we had in the old HTTP reference. That's done in just a few files:
https://github.com/bluesky-social/endpoints-standalone/blob/main/endpoints.config.ts configures the Lexicons that we're documenting on this site. In our case, this is
app.bsky,com.atproto,chat.bsky, andtools.ozone. Scoping this felt a little bit arbitrary at first — the goal here was to present the endpoints implemented by Bluesky-the-app while avoiding a whole schpiel about the ecosystem, but it matches what we had in our pre-lexdocs build, and, importantly, it means that any other atproto devs can copy this framework and update this one document to populate OpenAPI docs with any other set of Lexicon endpoints!https://github.com/bluesky-social/endpoints-standalone/blob/main/unimplemented.json is a list of endpoints from those namespaces that Bluesky the app doesn't implement. This was necessary because scoping the
com.atprotonamespace programmatically is pretty hard; even Bluesky has no hard requirement to implement everything in there, and the boundaries betweencom.atprotoandapp.bskyin particular are not really intuitive (which is why we present them side by side on the landing page).https://github.com/bluesky-social/endpoints-standalone/blob/main/src/build-openapi.ts is a build script that does almost everything else: hiding Scalar's cool inline endpoint tester for methods that are hard to mock up in a browser anyway, adding descriptive text and widgets for auth-required endpoints that need to be proxied through a PDS, putting
chat.bskyandtools.ozoneendpoints on another tab, et cetera. There's a bunch of cherrypicked logic in here to make our whole multiple-hosts situation more intelligible, but notably, it doesn't inject any other documentation that isn't already in the Lexicons, and it should be a totally replicable model for anybody else building (and documenting 🤓) in this ecosystem.
And that's about the size of it! I added some redirects to the docs.bsky.app site, yoinked out a ton of old content, and we were good to go. I want to thank everyone in the community for such a positive response to this unannounced improvement, and thank Scalar for maintaining a project that I can ship on a Friday without too much terror!
Hope you're all enjoying the World Cup (and the Knicks!!!!!!!) — more to share soon.