snarfed.org: Recent Episodes
None
Tomorrowland / Walt Disney
People regularly ask me whether Bridgy Fed is ready to scale and support more users. It’s a technical question, but their underlying motivation is usually broader: they believe in the social web, and the fediverse(s), and they want them to connect everyone who’s willing, across instances and networks and protocols.
Right now, the answer is, I don’t know. It’s not a technical thing; as an engineer, that part is catnip for me. I’m ready to roll up my sleeves and dive in. The more difficult part is organizational. Right now, Bridgy Fed is effectively one person’s side project. I love building and running (and funding!) it, and I have no plans to change that. However, it has basically no organization, governance, or institutional structure. It’s just me.
That’s ok! At least, as long as it continues to be one person’s side project. It’s growing, though, and people are starting to envision it, and bridges in general, as more important parts of the decentralized social web. Load bearing infrastructure.
Stable, reliable infrastructure is hugely valuable. To do it right, you need stable, reliable organizational structure. You need people to dedicate their time and expertise, sustainably. You need funding, and institutional governance, and some amount of transparency.
Right now, Bridgy Fed mostly doesn’t have those things. It’s one person’s side project.
That could change! I’m open to it. I don’t plan to lead that change myself, though. I’ve enjoyed building it in my spare time for many years now, and I have no plans to stop any time soon. It is not my career, though, or my calling, or my life’s work. I’ve spent my last 10+ years in leadership, I’m comfortable with it, but this isn’t where I’m personally looking to do it. For me, Bridgy Fed just a fun, hopefully useful side project. I’ve been between gigs for a bit now, spending a lot of time on it, but that won’t last forever. I expect to take a real job again eventually, and when that happens, I’ll have way less of that time.
xkcd: Dependency
So, to anyone hoping Bridgy Fed will become core infrastructure for the social web: that is one possible future! The first thing we’d need is an executive director or CEO, someone who wants to lead its organization, product, and policy. Someone who’d build relationships with groups like IFTAS, the SWF, Bluesky, IndieWeb, and others. Someone who’d own fundraising, if necessary. (Funding isn’t the real problem here, though. I self fund Bridgy Fed right now, and I could expand that to help with staff and other costs.)
This wouldn’t be a full time job; I expect it would only take 5-10 hours per week. It wouldn’t necessarily need a dedicated role or standalone organization, either. Bridgy Fed could live comfortably as one of many projects inside a broader group like IFTAS or the SWF, or even a benevolent company like Flipboard or non-profit like Ghost.
Another possible future for Bridgy Fed is the glide path it’s on now: one person’s side project. I can keep running it like this for the foreseeable future. Hopefully useful and stable, but definitely not core infrastructure. No real governance or institutional structure.
In particular, as one person’s side project, Bridgy Fed would probably remain opt-in in most places. This post is not about opt-in vs opt-out, or any other big policy or product decision, but it is about who makes those decisions, and how they should be made. Regardless of how public or global or searchable a network is, or how much it encourages tools to be opt-out – like Bluesky does – making Bridgy Fed opt-out anywhere would set more of an expectation that it’s core infrastructure. As long as it’s just my side project, I can’t satisfy that expectation.
If you think Bridgy Fed needs to grow up and be real infrastructure, and you’re interested in possibly leading it as executive director, or adopting it into a bigger organization, or you know somone who might be, that’s a very possible future. Drop me a line, I’d love to talk. In the meantime, when people ask me whether it can scale, or switch to opt-out, or what the long term plan is, I now have something to point them to. Thanks for reading.
Hi Bridgy Fed fans! It’s time for another status update, headlined by two big new features.
First, you can now use custom domain handles on accounts bridged into Bluesky! Web sites and fediverse accounts can both do this; click through for instructions. Here’s an example. We’ve been excited about this for a while, we hope it makes bridged accounts feel a bit more like first class citizens.
Second, if you deactivate bridging to Bluesky, you can now undo it and bring back your bridged account! Just un-block and re-follow the bot, or for a web site, file an issue or ping me. This is supported if you first deactivated after 2024-10-22; we’re still working on the rest.
I’m also excited that Tamschi has generously volunteered to help triage issues and coordinate development. He’s been a valuable presence on the project for a while now. Please give him a warm welcome!
Oh, and I got distracted for a bit and built a labeler that emits custom self-labels, then promptly realized that custom self-labels probably aren’t allowed, even though Bridgy Fed has been using them since launch. Ah well!
Lots more since last time too:
- Add original account ids to Bluesky DID documents
- Scale new user signup better
- Fix nasty rare bug that bridged reposts with the wrong original post
- Translate fediverse @-mentions to webmentions
- Better error handling in RSS/Atom feed processing
- Improve Sharkey/Misskey compatibility with images in posts and profile banners
- Improve GoToSocial compatibility (more)
- Detect and handle Bluesky handle changes better
- Bridge videos into Bluesky more efficiently
- Significantly improve Bluesky firehose efficiency
- Bluesky => fediverse: link mentions of non-bridged users to their bsky.app profile
- Improve handling of deleted fediverse accounts
- Full AT Protocol lexicon schema validation
- …and lots more bug fixes and internal improvements
As usual, feel free to ping me with feedback, questions, and bug reports. You can follow the now label on GitHub to see what I’m currently focusing on. See you on the bridge!
Future Publishing
I wish I had a better sense of where gravity comes from. Anyone have good intuition that they could explain?
I obviously get what it is: apple falls from the tree, etc. And I think I more or less get how it is: Newtonian mechanics, general relativity, curved spacetime, etc.
But I do not get why it is. Why does gravity exist? Why do large accumulations of mass naturally pull other mass toward them?
One theory I’ve heard is that there’s a universal field, like the Higgs, with particles traveling in all directions and colliding with mass. If you’re near a massive body like a star or planet or moon, it blocks some particles on its side from hitting you, but not from the other side, so the net effect pushes you toward it.
That would make gravity related to volume instead of mass, though, which obviously isn’t right. Still, I don’t remember hearing any other explanations. Does anyone know of any?
Added Farcaster to my social protocols comparison table. Click through for HTML version with text, links, etc. Feedback is welcome!
My sister makes her own greeting cards with a celery heart as a stamp. They turn out pretty great.
Discovered a nasty, days-old data bug in prod just now in an untested code path. Only hit a few users, but still, ugh.
Lesson learned: 90+% test coverage is great, but bugs still happen in the other 10%. It may be harder to test, but do it anyway. If you shipped it then you should have put a test on it.
Hi all! I’m overdue for another Bridgy Fed status update. Bottom line up front: we’re now bridging video on Bluesky! Out from Bluesky works now, in will work as soon as they finish their user rollout, hopefully within days.Also, Bridgy Fed can now ask people to bridge their accounts! This is the infamous “discoverable opt in,” and it’s finally launched. If you want to follow someone, but they’re not bridged, send their handle to Bridgy Fed in a DM or chat message, and it will message them to say you’re asking. (Only once; it won’t send another message if someone else asks.)
We send a couple other DMs now too, a welcome when you bridge your account, and an FYI when you reply to someone but they won’t see it because you’re not bridged .
It’s been a busy couple months. I was all set to post this weeks ago, after I put the finishing touches on DMs, but then Brazil happened. 3M new users and 10-20x usage increase in just a few days!
Amazingly, even though Bluesky team hosts all the difficult-to-scale parts themselves, not in the cloud, they’d still planned for this kind of surprise and handled it ok. Bridgy Fed though, not so much. It stayed up and serving, but Bluesky => fediverse got slower and slower. Tough timing, too, I was busy with other things and couldn’t find much time to work on it, so after a week it was almost 3 days (!) behind.
Fortunately, I finally managed to speed it up – props to Ilya‘s libipld library, among other things! – and after it worked through the backlog, we were back in business.
Anyway. Since last time: video, DMs, usage spike, and other features and bug fixes too:
- Improved Pleroma and Flipboard compatibility.
- Improved style of web => Bluesky bridged posts, also now try harder to include an image.
- Drastically improved monitoring and alerting.
- Stricter authorization for AP
Undos. - …and lots more bug fixes.
As usual, feel free to ping me with feedback, questions, and bug reports. You can follow the now label on GitHub to see what I’m currently focusing on. See you on the bridge!
It’s been a month since the last Bridgy Fed status update, so it’s time for a new one! I’ve spent a lot of time over the last month on user-visible improvements and bug fixes, notably more complex post types and links, as well as underlying infrastructure. Details below.One area I need to spend more time on is cost. My attempts at optimization there have been slower than I’d originally hoped. I intentionally prioritized functionality over cost efficiency for a long time, and I’m still optimistic that I can get cost per user down to a reasonable level. If I do that, but user count itself gets too high…that will be a wonderful problem, and I’ll cross that bridge when I get to it.
(Standard disclaimer: Bridgy Fed is non-commercial, free, open source, and ad-free, and I have no plans to change any of that or ask for donations any time soon! It’s one way I try to support and give back to the open social web.)
Beyond cost, I’m hoping to work on native replies and opt in prompts via DM, both directions, soon.
As usual, feel free to ping me with feedback, questions, and bug reports. You can follow the now label on GitHub to see what I’m currently focusing on. See you on the bridge!
- You can disable and delete your bridged account, on either side, by blocking the Bridgy Fed bot user (background). Note that right now this cannot be undone. Maybe eventually!
- Lots of improvements to rich post/profile content and text:
- quote posts
- videos and GIFs
- content warnings
- link previews
- hashtags
- @-mentions (more)
- profile bios
- full text of truncated posts and profile bios for third party Bluesky apps like Skythread and Skywalker
- …and other bug fixes
- Improved authorization/access control checks
- Infrastructural improvements:
- ATProto PDS XRPC methods and error handling
- webfinger HTTP status codes
- firehose client stability
- firehose server stability
- better HTTP caching
- Cut new releases of arroba, lexrpc, and dag-json, three libraries of mine that are key building blocks for Bridgy Fed’s Bluesky (AT Protocol) support.
Finally got around to reading Geoff Huston’s comprehensive, heartbreaking eulogy for DNSSEC. He takes the thought-provoking position that TLS has largely subsumed the need for DNSSEC – not perfectly, and with a somewhat different threat model, but still. Great networking neckbeard reading, as always.
Bridgy Fed status update time! It’s been a busy couple weeks. I’m happy to report that I’ve made solid progress on three of my four goals from last time – reliability, delays, and bugs – and improved a number of other things along the way. Here’s a sampling:* Switched Bluesky => fediverse from polling to firehose. This made it way less flaky, and cut the 5-15m delay down to just seconds. * Fixed fediverse => Bluesky getting stuck for many accounts. I took some shortcuts in my Bluesky infrastructure, and they came back to bite me. Still work left to do, but the user-visible issues should be largely fixed. * Various issues with finding the bot accounts, following them, and getting bridging started. * Fixed images in Bluesky => fediverse posts. * Implemented deleting posts, Bluesky => fediverse. (Fediverse => Bluesky was already working.) * Fixed flakiness bridging replies, both directions. * Fixed links, mentions, and hashtags, both directions. Still some broken bits here, notably Bluesky => fediverse mentions, but otherwise these should be much better now. * First pass at spam filtering. Sigh.
Apart from the firehose, one notable area I haven’t focused on yet is infrastructure. It’s scaling ok so far, but it could be a lot better, and a lot more efficient. I’ll need to work on that soon. If you see a bit of time go by without many user-visible improvements, that’s probably what’s happening.
Otherwise, thanks for all the feedback and questions and bug reports! Please do keep them coming. As always, you can follow the now label in GitHub to see what I’m focusing on. See you on the bridge!
I’m at my kid’s volleyball game, and I just watched her accidentally high five one of her teammates in the face.
Psyched to play Animal Well! The descriptions so far are giving me very NetHack/Rogue vibes. Anyone else?
Phew! It’s been quite a few days, Bridgy Fed is up to almost 2500 bridged accounts total. Thanks again for all of your interest! I’ve been in a constant state of whiplash, back and forth between hearing “this is so cool!” and “it’s broken in these 10 different ways.”I love that you all are enjoying it! And it absolutely is broken in all of those ways, and more. I’m sorry. I’d planned on a bit more time to iron out bugs first, but c’est la vie. Testing in production!
I’m seeing most of the problems break down into four broad buckets, which I’m tackling in this order:
- Bridging stuff from Bluesky to the fediverse is flaky and unreliable. True. This is largely because it’s currently based on polling. I’m in the middle of switching it to the Bluesky firehose; that should hopefully be complete within a day or two.
- Bridging stuff from the fediverse to Bluesky sometimes stops working. True. Bridgy Fed isn’t as rigorous about validating Bluesky records as it needs to be, so when we emit an invalid record in one user’s repo, I suspect the Bluesky relay gets stuck and won’t accept any more for that user. Fixing some parts of this may be tricky, but it’s still very fixable.
- Lots of things don’t translate right, or at all. True. This is annoying, but seems lower priority than the unreliability buckets, and most of these bugs are straightforward to fix, so I’m focusing on reliability first.
- Scaling is inefficient and expensive. True. OK, I’m the only one seeing this, but it’s still true. Fortunately I’m not too worried, since I’m confident I’ll be able to optimize it. Make it work, then make it right, then make it fast (and cheap), in that order.
I’ll continue to post updates here now and then. You can also follow the now label on Bridgy Fed’s GitHub issues to see what I’m focusing on at any given moment. Thanks again for all of the interest, and for your patience. It’ll get better, one step at a time.
there are few things quite as glorious as a whole day with no meetings
we all know maker’s schedule manager’s schedule, but still, the impact on productivity never ceases to amaze me. way bigger than linear
(even so, I do miss the bandwidth of a whole team! way harder and slower doing everything myself)
Hi all! Well, I didn’t plan it, but word got out anyway: Bridgy Fed‘s Bluesky <=> fediverse support went online a couple weeks ago, quietly and without announcement, but people still found it. Over 1200 accounts have turned it on so far and bridged themselves one direction or the other. It’s great to see so much early interest!
(For anyone unfamiliar with Bridgy Fed, here’s an introduction.)
It’s still very early, and I wasn’t quite prepared for this much volume this quickly. Expect lots of bugs, missing features, downtime, and other rough edges. The docs are mostly up to date, at least. If you hit a problem, search the open issues, and if you don’t see it there, feel free to file it.
Also, much of the current state is not final. Expect some design and policy choices to change. For example, right now you have to manually enable the bridge, but that may change eventually, at least for Bluesky accounts bridging into the fediverse.
Otherwise, there’s a ton to do, and I’m only one person, doing this on the side, so progress will be slow. They say if you don’t feel uncomfortable when you ship, you probably waited too long, so let me just say that I feel very uncomfortable right now, hopefully in a good way. Thanks again for all the interest, and for your patience!
Hi all! It’s been a while since my last status update on Bridgy Fed, its upcoming Bluesky/AT Protocol support, and the resulting firestorm.
It’s coming along! It’s not launched yet, I still have a number of things to build and tests to run, but it’s getting there. Also, Bluesky’s current federation test is limited to 10 users per federated server. We can’t launch until they lift that limit. I don’t know when that will happen, but I’m confident it will.
I’m also blocking launch on building the opt in/out prompt idea that came out of the blowup. Thank you all (and Kio!) again for that idea, I really do appreciate it. I wish I’d come up with it beforehand, would have saved everyone the headache. But it seems like this – getting feedback before launch and incorporating it – is the system working, at least in some ways, which is good.
The current design is that a Bridgy Fed instance actor (user) will DM you the first time anyone on Bluesky requests to follow or interact with you over the bridge. If you reply yes, or follow the Bridgy Fed user, you’ll be bridged. If you reply no, or ignore the DM, or block that user, you won’t be. You’ll also be able to follow or block the Bridgy Fed user to opt in or out proactively, ahead of time.
On a related note, I still think there’s a tension between the fediverse’s current default of open, opt-out federation and its culture of consent and opt-in. That tension is magnified by the fact that the fediverse has always been multi-protocol, not just ActivityPub but also OStatus, Diaspora, and Zot/Nomad, among others, so boundaries between networks have been fuzzy at best.
For people who want it, consent-based/opt-in federation is the most promising solution that I’ve heard so far. Beyond that, I don’t have any answers of my own, but there’s obviously been lots of discussion over the last couple months, which feels like a good sign.
As always, feedback is welcome!
Tamagotchis sure have come a long way. Kid proudly informed me just now that she doesn’t need to bring hers with us running errands because she hired a babysitter for it.
One of our cats purrs so loudly that it’s a running joke in our family. It makes her fur vibrate. We have to turn the TV volume up when she sits with us. Once, she was in another room, and I heard her purr through the wall. No joke.
Love this account of taking one of the first photographs in history.
Everyone is acquainted with the beautiful effects which are produced by a camera obscura, and has admired the vivid picture of external nature which it displays. It had often occurred to me, that if it were possible to retain upon the paper the lovely scene which thus illumines it for a moment, or if we could but fix the outline of it, the lights and shadows divested of all colour, such a result could not fail to be most interesting. And however much I might be disposed at first to treat this notion as a scientific dream, yet when I had succeeded in fixing the images of the solar microscope by means of a peculiarly sensitive paper, there appeared no longer any doubt that an analogous process would succeed in copying the objects of external nature, although indeed they are much less illuminated.
Not having with me in the country a camera obscura of any considerable size, I constructed one out of a large box, the image being thrown upon it by a good object glass fixed in the opposite end. This apparatus being armed with a sensitive paper, was taken out in a summer afternoon and placed about a hundred yards from a building favourably illuminated by the sun. An hour or two afterwards I opened the box, and I found depicted on the paper a very distinct representation of the building, with the exception of those parts of it which lay in the shade. A little experience in this branch of the art showed me that with smaller camerae obscurae the effect would be produced in a smaller time. Accordingly I had several small boxes made, in which I fixed lenses of shorter focus, and with these I obtained very perfect but extremely small pictures; such as without great stretch of imagination might be supposed to be the work of some Lilliputian artist. They require indeed examination with a lens to discover all their minutiae.
In the summer of 1835 I made in this way a great number of representations of my house in the country, which is well suited to the purpose, from its ancient and remarkable architecture. And this building I believe to be the first that was ever yet known to have drawn its own picture.
- Some Account of the Art of Photogenic Drawing, or the Process by which Natural Objects may be made to delineate themselves without the aid of the Artist’s Pencil, Henry Fox Talbot, 1834
via Pandaemonium 1660–1886, Humphrey Jennings
“Papa, will I need to know how to get the circumference of a circle if I want to be an author?”
Started to help the kid with math homework tonight and realized I completely forgot how to do long division.
Threw together a comparison of the four decentralized social protocols I know best: IndieWeb, ActivityPub, ATProto, Nostr. Obviously oversimplified, hopefully still useful! Preview below, click through for full table with links.
I tried to focus on how these protocols are currently deployed and used in the real world. For example, identity in ActivityPub is technically URL-based, but in practice the fediverse uses WebFinger user@domain identifiers more or less universally, so the table reflects that.
Feedback is welcome!
What are the origins of social networking? If we restrict ourselves to online, are they .plan files? Usenet newsgroups? BBSes? Minitel? Comments in code on floppies passed around by mainframe programmers? Or do we consider offline too? Christmas cards, phone trees, 17th century pamphlet wars? Salons? Roman speeches in the Forum? (I’m hoping not to […]
Hit a big milestone in AT Protocol development today: my PDS implementation arroba is successfully federating with the sandbox! That means its repo, MST, commit chain, sync protocol, and more are all working and interoperating. Took lots of work to get here, feels good to have it all come together. Lots more to do before […]
Having spent a fair amount of time developing for all of them, my experience has been that most proprietary social network (“silo”) APIs are on the easy/low effort end of the spectrum, followed closely by the Bluesky API, webmention + microformats2, and Nostr, then a big jump up to ActivityPub at above average, then AT […]
8 yr old just now: “I washed my face and hair! Know why? It gives me fresh minds, fresh looks, and fresh ideas!”
I’m no music critic, but I think Imogen Heap‘s Hide and Seek may be one of the best popular music songs of all time. Right up there with American Pie, Hallelujah, Thriller, etc. Definitely the vocorder’s pièce de résistance, I don’t know if anyone else has taken that instrument quite so seriously before, or since.
I’ve written before about how your fediverse identity, data, and administration are all tied to your instance, for both technical and cultural reasons. There’s an awkward corollary: functionality in the fediverse is also currently tied to your instance. Signed up on mastodon.social? Great! You can microblog…but you can’t really do groups, long-form writing, live streaming, […]
We outdid ourselves with the seasonings on the salmon this year, each from a different person. Top to bottom: orange ginger garlic, Korean BBQ, cedar plank, lemon dill, and the pièces de résistance: crushed Cheetos and Flaming’ Hot Cheetos. Both surprisingly good! Cheetos tasted oddly like (good) ground beef, Flamin Hot was rich and multilayered […]
Unpopular opinion: I have deep reservations about policing misinformation. I’m a big believer in the demarcation problem: in the general case, there’s no shortcut to determining objective truth. It’s similar to the halting problem in computer science: if we want to know whether a claim is true, we have to go through the scientific method. […]
OH at the golf course: “Par, baby! Woohoo! Fuck yeah!” Good to have goals in life, I guess.
I was traveling in Hong Kong once, ages ago, when I ran out of toothpaste. I went to a convenience store, grabbed some, and when I went to check out, the clerk looked confused. She pointed at her teeth, then at me, in surprise. She didn’t know English, and I didn’t know Cantonese, so I […]
I’m always amused by the contrast between big companies’ investor statements – “We’re awesome! At everything! And we’re getting even more awesomer!” – and their SEC filings and antitrust statements – “We suck! We’re a widdle tiny helpless baby, in a huge scary market, waaaahhh…” Microsoft is the obvious current poster child, but it seems […]
Icky horrible thought of the day: what do we think about AI-generated child porn/CSAM? On one hand, it’s horrible, and icky. On the other hand, as an alternative to actual real CSAM, the harm reduction seems huge. On the other other hand, are there other dangers to allowing this kind of material to proliferate more […]
Well, well, well, what do we have here. It’s this year’s Bridgy stats update, and it’s one of the most eventful in a long time. Notice anything new?
That’s right, there’s a huge ass cliff a couple months ago. Let’s see that a bit closer….
If you guessed that cliff was Twitter, ding ding ding, winner winner chicken dinner. The Twitter API apocalypse came and went, and they still haven’t managed to found the off button itself – some smaller apps like granary are still up and running just fine – but they manually suspended Bridgy and most other nontrivial apps.
Twitter accounted for a majority of both Bridgy accounts, 4k of the 10k total as of April 2023, and backfeed and publish volume, which you can see in the plateaus in sent webmentions and successful publishes over the last couple months.
The other striking trend is the marked uptick in Mastodon account growth since Elon took over back in November. It’s more than doubled in the last 6 months, from 557 on Nov. 1 to 1125 yesterday! Exodus indeed.
In Bridgy land, the Twitter implosion and Mastodon growth has spurred more development on granary and especially Bridgy Fed, its decentralized/federated cousin, than on Bridgy itself. Expect that to continue for the foreseeable future.
As for the rest of the numbers: 6278 accounts (after subtracting Twitter), 32M silo responses, 5.3M links, 2.5M webmentions sent, 125K posts published, 138K blog posts processed, 18K blog webmentions received. Apart from Twitter and Mastodon, trends and growth and costs are largely stable since last year. The uptick in the “other” (ie not Bridgy) yellow line is largely due to Bridgy Fed growth.
Data,methodology,previously,previously,previously,previously,previously,previously,previously,previously.
Hey decentralized social people, can I ask a favor? Here’s a rough draft of how I’m thinking about translating/bridging across multiple protocols and identities (via federation, ie not syndication/cross posting). Could you take a look when you get a chance and see which parts look reasonable and which are totally off base? Feel free to comment here or on the PR, whichever’s easier.
Hugely appreciated, thanks in advance!
Colorful monitoring graphs always remind me of the Museum of Borgmon Abstract Art. Such an underappreciated gem! Really glad they finally made it public.
I don’t spend much time on software engineering industry reports, but the Thoughtworks Radar is one I love. Broad, clear-eyed, hands on, eminently useful. Bite-sized analyses of ~100 tools and techniques, new and old, with recommendations to adopt or trial or assess or hold, and a brief nod toward overarching themes. It’s evidently become so successful that it’s spawned an entire category.
The latest volume came out last month, and as usual, it’s a banger. I’m only maybe a third of the way through, and I already have todos to learn more about run cost prediction, accessibility testing, and Logseq, among others.
Free, ~40 pages, twice a year. Not an ad, I have no affiliation with them, just a fan.
remembered an important fact about myself just now
deep down, at heart…I’m a normie
(and I’m ok with that)
One core difference between the fediverse and the AT Protocol seems to be that AT decouples many key building blocks – identity, moderation, ranking algorithms, even your own data to some degree – from your server. The fediverse, on the other hand, ties them all to your server and sees that as a desirable feature.
The fediverse says, choose a server that you identify with as an individual, with admins who moderate according to your values, and a local timeline that you like reading.
The AT Protocol, on the other hand says sure, choose all of those, but independently from your server, and keep those choices if/when you migrate to a new server.
Interestingly, I don’t think much of this is really driven by ActivityPub itself except identity. Third party AP moderation tools could easily be built, and probably have been. Same with clients that rank your feed with custom algorithms. This seems like more of a cultural difference, a difference of values and philosophy about how social networking should work.
the correct threat model for things you post to the public internet is “it’s constantly being scraped and persisted”
– @dystopiabreaker.xyz
if you can attach a chainsaw to a robot vacuum cleaner, I believe you have a moral obligation to do so
Data sizes, a taxonomy:
- Tape backup facility
- Spark + Parquet cluster
- Fits on disk
- Fits in memory
- Just use a list
so I’ve never really been Very Online, and these last few months on Bluesky have been by far the most time I’ve ever spent around shitposting, weird twitter, tpot, whatever it is
can I just say…it’s a lot
Richard Dawkins coined the term meme in his 1976 book The Selfish Gene to mean "a unit of cultural transmission, or a unit of imitation."
I wondered what he’d think of it evolving to mean primarily "silly bad internet joke" these days. Evidently he told us what he thought, somewhat, back in 2012:
It is very good that Wikipedia also gives a page to "Internet Meme". Internet memes are arguably the most important subset of memes today.
PetMD
…or if you prefer, eating my own cooking, or scratching my own itch, or drinking my own champagne. Sure. These are all metaphors for the idea that if you build something, it turns out better if you use it yourself, especially if you want it yourself. However, when I think about my projects for bridging social networks, I wonder if I don’t use them myself deeply enough. If I’m not the target audience. Is that a problem?
It’s not entirely true. Strictly speaking, I do use them. After this post gets published, you’ll see a trickle of likes, reposts, and replies from social networks start to show up down in the comments, thanks to Bridgy and Bridgy Fed. The part I worry about isn’t the tools part, it’s the social part. How online social tools should work, how communities should use them, how they affect the ways people interact online. These have all been hot topics for a while now, with social networks pushing “healthy conversations” and Congress haranguing tech execs on Capitol Hill, and even more acutely recently now that Twitter is burning and a new crop of social networks has sprouted.
These questions are complex, deep, and important. Many people have their own slants: big companies on business models, startups on features and news, IndieWeb on owning your data, the fediverse on consent and safety, libertarian techies on anti-censorship, government officials on…whatever helps them get re-elected, I guess.
I don’t know which of these angles is right, but I do know the issues are important. And as someone building social plumbing and tools, I’m keenly aware that my choices directly impact them, if only for my relatively small user base. They’re not easy choices! In Thorsten Ball’s dichotomy, I’m fully type 2: if a technical problem requires human behavior, that makes it more difficult to handle, not less.
Curioso
The problem is, I don’t have my own angle. I don’t know how tech should handle online social interactions – granted, probably no one does for sure – and I’m not particularly qualified or motivated to tackle it. Part of it is that I don’t really hang out on the internet. I’m somewhat online in a few bits of open source, but only somewhat, and not a ton elsewhere. I don’t post many times a day, I’m not on Twitter or Mastodon for hours at a time, I haven’t made many close friends on the internet. I definitely don’t have deep experience in community organizing or support.
At the same time, I’m not under any illusion that the tools and services I build are neutral. We’ve mostly matured beyond “tech isn’t good or bad, it’s how it’s used,” especially for social tech. Joel Spolksy’s historical view on this is one of my favorites, including his “primary axiom of online communities”:
Small software implementation details result in big differences in the way the community develops, behaves, and feels.
I’ve seen this firsthand with Bridgy. Most users love it, but I do occasionally hear complaints that it creates surprising context collapses when someone’s reply shows up in a different place than they originally posted it. There’s also the broader concern that webmentions support and promote public conversations over private ones, and the ongoing debate over whether they hurt or help your control over your own data. These conversations are many years old, but the recent explosion of alternative social networks and the fediverse has injected new life into them.
Again, these are important questions. We need to figure out how to design healthy online spaces and tools! And I may have a few loose opinions here and there, but in general, I don’t have deeply held ideas or convictions, nor do I have a burning desire to work on the problem. It’s just not me.
I’m grateful to the people and groups who are. And honestly, I’m not too worried. I still believe I can build tools that are net positive even if I’m Not That Online. I don’t feel too much like I’m neglecting some internet civic duty. But every now and then, I wonder if I’m not eating quite enough of my own dog food cooking, or not in quite the right way, or something.
Am I overthinking it? What do you think?
date night. kid spent a few hours with the high school babysitter. both girls, one 8, one 14-ish. what did they do? first, painted their nails. second, coded a video game.
2023, folks, gotta love it
More and more these days, when I mindlessly grab my phone, I’m not finding anything in the dopamine drip feeds. Email, messages, socials, blogs, Slacks, news…all empty, unless it’s been hours.
I think this is good? A sign that I’m gradually dialing it in and pruning down to just stuff I actually consciously choose, which turns out to be not much, and not often? I don’t feel like I’m missing a lot, if anything, so that’s encouraging.
The last 200 years or so were a brief, glorious blip in history when we could use recorded media – photographs, audio, videos – as evidence or proof of anything. Barely a blink of an eye, relative to the full history of civilization. Nice while it lasted, guess it’s over now.
Well, it’s come to this. Twitter is burning, a billionaire owes money, an API will soon get lobotomized, so Bridgy‘s Twitter support will die within the month. Granary‘s and twitter-atom too. The Twitter API may now be effectively unmaintained, but they still managed to find an engineer somewhere to change a few numbers in the billing code and update some text on a web page.
What a waste. Plenty of ink has been spilled on all this already, I won’t belabor the point, but what an utter waste.
Right now, Bridgy uses a free tier of Twitter’s API, equivalent to what many other major social networks offer. By April 29th, this free tier will disappear. If I want to read tweets, my options will be a $100/mo plan with a quota of 10k tweets/mo, roughly .1% of what Bridgy currently uses, or an enterprise plan with unknown quota that reportedly starts at $42k/mo.
Neither of these options is feasible. Bridgy can’t function with .1% of its current usage, and I won’t pay Twitter $500k/yr for a little side project.
The silver lining is, after all the chaos and destruction and flight to the fediverse, Twitter doesn’t feel nearly as important now as it did half a year ago. It’s always been Bridgy’s biggest user base, it had a great 11-year run, I never quite expected it to end like this, but here we are.
To everyone who used it, thank you for your interest and support over the years! It’s been a great ride. And who knows, Elon’s Twitter 2.0 is awful at comms and changes its plans all the time, so there’s a chance they’ll take this all back tomorrow. But assuming it sticks, Bridgy Twitter will stop working on April 29, if not before. I plan to leave it up and running until then.
A billionaire owes money, Twitter is burning, and the future is the IndieWeb and the fediverse. Try out Mastodon with Bridgy, classic or Fed, and join us there!
Amidst all the excitement around Twitter open sourcing its “algorithm,” one bit is easy to miss: they licensed it as AGPL. That means anyone else who uses it will have to open source their code too, even cloud-based web sites and services, but Twitter themselves are under no such obligation to open source the rest of their code. Clever!
Pouring one out for Gordon Moore, what a legend. Truly one of our forefathers. We’ll all be forever in your debt.
Matilda, 1996
Content negotiation is a feature of HTTP that lets clients ask for, and servers return, different content types based on the request’s Accept header.
Sounds great, right? Well, no. Content negotiation is the classic example of an idea that sounds good in theory, but for the vast majority of web developers, turns out to be net harmful in practice. A few reasons:
- Most web developers are not aware that conneg exists. This is usually fine, since it’s uncommon, but can be an unpleasant surprise when they first hit it and don’t understand it.
- The
Acceptheader is notoriously complicated to parse, generate, and fully comply with. Imo much of that complexity is unneeded in practice. - It often breaks caching when it’s first introduced. Internal and external caches generally ignore Content-Type and Accept (etc) by default, so when conneg is first introduced, clients often get the wrong response type. Developers learn the hard way that they need to add Accept to Vary and to their framework’s cache.
- URLs leak across contexts. They may start out within a single service or API, but they gradually proliferate into user-visible links, object identifiers in other systems, add-on services, third party tools, etc. These generally don’t know the original conneg requirements, which causes unpleasant surprises.
I think most of this boils down to: modality generally considered harmful. When something always behaves the same way, it’s reliable and easy to use. When it behaves differently based on something far away that you may not know exists, it’s unreliable and surprising. Add in a very large ecosystem of independent tools that all need to interoperate, often in fine-grained ways, and you have a recipe for failure.
Europe is the weirdest continent. I mean, Australia? Sure. Africa, North America, South America, definitely. But why does this awkward little spiky corner of Asia get to be its own whole thing, geographically? Weird.
World Map Blank
Human beings are both novelty seeking and change averse. We want everything to be both new and familiar, at the same time. It’s the worst.
single entendre
(noun, humorous)
A phrase that has a single, often bawdy, meaning and is lacking in subtlety or cleverness.
Reading the Urbit whitepaper and this gem stopped me cold:
Nock is permanently frozen and will never need updating.
I guess you have to admire the chutzpah, if nothing else?
A long time ago, I decided to show Bridgy‘s end users its raw logs. Like, raw logs. HTTP requests, database reads and writes, JSON objects, stack traces, etc. It’s an unusual UI feature, but it’s been an unqualified success, enough that when I built Bridgy Fed, I immediately included it and never looked back.
Whenever Bridgy does something nontrivial – poll a social network account, send a webmention, publish a post – I generally include a link to the server logs for that operation. Here’s an example, a series of timestamped plain text log messages from a poll of my Twitter account. They include initial config and parameters, account status, each individual Twitter API request, the results of those requests, how Bridgy interpreted them, HTTP requests to my web site, the subsequent actions Bridgy took and why, how the account’s status changed, and when the next poll is scheduled for.
These kinds of logs can answer a number of common user questions:
- Why isn’t Bridgy seeing this post?
- Why didn’t Bridgy send that webmention?
- Why isn’t Bridgy interpreting my microformats right?
- Why did Bridgy’s request to my web site fail?
- Why did Bridgy’s request to that social network fail?
- What exact data did Bridgy send in this webmention?
- What exact data did Bridgy send to that social network?
- Look what Bridgy did here, is this a bug?
- etc.
Why do I like this so much? In a word, ROI. I’m not a UX designer or frontend engineer, I dread building UI, so I’m always on the lookout for ways to minimize it or avoid it altogether. Unfortunately, Bridgy is a complex tool. It connects to dozens of external services and thousands of web sites, it has complex internal logic in a number of ways, and it requires involved, specific setup on users’ parts. I work hard to keep the docs complete and up to date, but in tech, we all know that users don’t read.
So, as a way to communicate a broad, deep range of complex information, logs give me tremendous bang for the buck. Plain text log messages are flexible, cheap, and widely supported in most infrastructure. I already include logging during development as an observability tool for myself. I had to build an extra handler to serve those logs to end users, but that was mostly straightforward. Mostly.
This isn’t for the faint of heart though. It’s a tricky idea with a number of drawbacks:
- It only works for relatively technical user bases. Plain text logs are not good general-purpose UI. Wading through a wall of fixed width text and technical jargon is akin to seeing the matrix. Bridgy’s user base is relatively technical, but that’s not the norm. For most mainstream users, it’s a non-starter.
- Furthermore, even if most of your users are technical, the fact that logs are bad UI means they may cause confusion in the minority who aren’t.
- Exposing raw logs is risky! Most services have credentials, private data, and other non-public information. Developers often try to keep those out of logs, but some always slips in. That’s not critical when logs are internal only, but as soon as you expose them publicly, any exposure is unacceptable. You have to sanitize all non-public information, or at least anything that the logged in user shouldn’t see. That work is tedious, detail-oriented, and error-prone.
- Similarly, logs air all of your dirty laundry. All those quick hacks, shortcuts, missing features, old legacy incompatibilities, internal design decisions, and straight up bugs are on full display for anyone willing to spend the time to look for them. This can be good, transparency and all, but still. You have to be ok with the world looking through your underwear drawer and second guessing everything you do.
Regardless, even considering all of those tradeoffs, I love the logs UI. It’s saved me – and users! – countless hours of support and debugging. Consider it next time you build something for developers or other technical people. As low effort, high return, “worse is better” UI goes, it’s one of my favorite techniques.
Hey, thanks for the kind words, and for the idea! Bridgy is indeed busy, but it’s had time to evolve a pretty scalable architecture, so load isn’t really a problem. Some details here, including costs. I actually increased its poll frequency the other day, which doubled its load, and it’s still fine.(I do love GitHub Actions! But it’s more for CI, CD and other heavy, infrequent, repo-based tasks. Bridgy’s task volume is much higher, and the tasks themselves are lighter, so its current task queue probably fits better.)
A blog post on the results of the 2022 Emacs user survey. The first half explains that it took an extra year because they didn’t like any of the existing FOSS survey tools, so they wrote their own. In Julia.
Learned about a new job today: intimacy coordinator. Fascinating! I’d never heard of it before. What a world we live in.
(Via this interview with Manon Gage, star of the formidable new game Immortality, which I just finished. Incredible experience, a work of art.)
Re-launched instagram-atom, my side project that lets you read your Instagram feed in any feed reader, with new browser extensions for Chrome and Firefox. Feel free to try it out, feedback is welcome!
All the recent Twitter drama has obviously sparked renewed interest in Mastodon and the fediverse, and that’s even included Bridgy Fed, my little IndieWeb side project that turns personal web sites into full-fledged fediverse accounts. A month and a half ago, I was all set to take it easy, maybe play a game or two, […]
So Mastodon 4 went js;dr, which means it requires JavaScript to render content. That means that server-side fetches of Mastodon 4 user profiles and posts, eg indieweb.social/@snarfed, no longer return the actual contents in the response. They definitely no longer include microformats. 😐 JavaScript and SPAs are their own conversation, but regardless, this is a […]
Thinking of Fred Brooks, who passed away yesterday. What a legend. Mythical Man-Month alone is one of the foundational texts of our field, not to mention his other work. Fred, you will be remembered.
Xfinity WiFi Hotspots is a great product. I can walk into pretty much any coffee shop or bar or park, open my laptop, and I’m immediately online. No begging baristas for a password, no sketchy FREE PUBLIC INTRNET 4 REALZ access points, no fiddly tethering to my phone. So nice. (Yes, Comcast the company. I […]
A while back, early in the blockchain hype cycle, a startup called Verisart popped up and promised to “fix” fine art. Not sure if that painting is real? Can’t find out who owns it, or where they got it? Worried that the gallery you’re emailing is a scammer? Worry no more! Verisart puts it all […]
Bought snarfed.eth. Not sure why. Might be fun to play with? But I also feel kind of dirty now, like I need a shower. At least it was only $30, not $150 like a year ago.
I’ve been hearing and saying “don’t let the perfect be the enemy of the good” a lot over the past few days. The only way I know how to make something good is to ship and iterate. Get something usable out there, in front of yourself and eventually others, use it (read it etc), get […]
I’ve added a significant new Bridgy Publish feature: Micropub support! Up until now, Bridgy Publish offered two ways to post to silos: interactively from user pages and via webmention trigger. Now there’s a third way: Micropub. You can now connect a Micropub client to Bridgy and use it to publish posts to any supported silo. […]
Finally took the time to learn homomorphic encryption. Wow. Mind blown. Modern cryptography is magic!
ipv4.games is an utterly brilliant neckbeard nerd snipe. (Oh, sorry, silly me, I meant ipv4.games, of course.)
One key thing we’ve learned about the human brain is that it’s just like the rest of the body: every part of it can go wrong, in a million different ways. It’s unlikely that the gender identity part is the one special case that always works perfectly and is 100% male or female.
"Two powerful and opposing drives — the desire to enjoy the world and the urge to set the world straight." - E. B. White https://en.wikipedia.org/wiki/Walden#Background_information
TIL the famous Chinese curse “May you live in interesting times”…isn’t actually Chinese at all.
Love to see this: no-touch security updates for consumer OSes. About time too, servers have had them for a while, unattended-upgrades etc. Now do Windows and Android and browsers. 😎
It’s that time of year again…Bridgy stats update! Top line numbers: 6471 accounts, 28M silo responses, 4.7M links, 2.3M webmentions sent, 99k posts published, 102k blog posts processed, 16k blog webmentions received. Trends and growth are mostly stable since last year. One noticeable exception was a sharp uptick in publish attempts that made me split […]
Global average lifespan in 1900 was just 32 years old. 32! It’s now 73, more than double, just a century and change later. I knew it had increased, but still, staggering reminder. Just one of many ways the world has improved massively in recent history.
First pianissississimo I’ve ever seen in the wild, maybe. Not sure my fingers are physically capable of playing that quietly. (Rachmaninoff, Barcarolle, Op. 10 no. 3, top of page 5.)
When you hear “I do my own research,” translate to “I search for random people on the internet who will tell me what I already believe, in greater detail.”
The division of powers among the layers of civilization allows us to relax about a few of our worries. We should not deplore rapidly changing technology and business while government controls, cultural mores, and so-called wisdom change slowly; that’s their job. Also, we should not fear destabilizing positive-feedback loops (such as the Singularity) crashing the […]
Old San Francisco postcards, probably early to mid 20th century. Discovered at Dickens Fair many years ago.
Kid: Can I watch squirrel video?Me: Sure, after you do your chores for the day.Kid: How many chore skips do I have left this week?Me: I think you’ve used them all.Kid: [thinking…] Can I use one from next week?Me: Clever! You’ve invented the advance. Well…sure, why not.Kid: [thinking more…] Could I pay money from my […]
Matthew Childs / Reuters We live in a golden age of software reuse. We’ve never before had such a wealth of freely available code, in so many languages, so easy to find and install. And yet, we’re drowning. We slap together rickety rowboats and toss them out on PyPI Ocean and npm Sea, then act […]
Took the 7 year old to Disneyland for the first time. Plenty of new stuff since I was there last, almost 20 years ago, but I love how much of the original park is largely unchanged. Main Street, Castle, Small World, New Orleans Square, Tiki Room! Still a magical place, especially seeing it through a […]
Had a big new project idea just now. Deep, compelling, fun to build, fun to use, may not exist yet. Added “research X” to my todo list, tabbed to the notes field, couldn’t type fast enough to keep up with all the follow-up thoughts and ideas. Only happens to me every few years or so. […]
Today marks 10 years to the day since I first launched Bridgy, my little IndieWeb side project to connect social networks and personal websites. Happy Birthday, Bridgy!
I’ve always loved the internet, but I’m not a Very Online person, exactly. I don’t really hang out there. I didn’t fall in love with the people, or the community; I fell in love with the network. The physical reality of packed switched routing, the awkward unlikely miracle of a bunch of computers – rocks we flattened and jammed lightning into – talking to each other. Interacting. Just a few at first, here and there, but gradually more and more, on long strings of copper underground and hung on wooden poles and finally across entire oceans, until they circled the globe in a single, all-encompassing internet. Kind of amazing.
I still love all this. My favorite guilty pleasure reading material is potaroo.net, Geoff Huston’s fantastic blog on internet plumbing from deep in the bowels of APNIC. His deep dives into BGP, IPv4/v6, DNS, and the rest of the modern networking world scratch an itch for me that nothing else can touch. They’re great.
Naturally, I’ve always wanted to participate in the internet too. First BBSes and early email, then walled gardens like CompuServe, then network programming in high school and college, and finally here on my own domain. I feel like my web site is a home, a home I own, in a way I can’t say about my Facebook or Twitter profiles.
I’m not the norm, though. Most people are online these days, but not on their own web sites or domains. It’s not as hard as it used to be, but it’s still not easy enough. How many of your family or friends have web sites? Probably not many. How many are on Facebook or Twitter? Many more.
I had my own web site, and I posted to it on and off. After a while, I started copying those posts into the social network silos, because that’s where everyone was. I didn’t mind, and people still occasionally came here and commented, but more and more, they did it in the silos instead. Those responses felt ephemeral, fragmented. I didn’t really own them, any more than I owned my silo profiles.
I knew how to code, and the social networks all had APIs, so I dove in. I’d followed the federated social web movement for a while, so I knew about OStatus, the main open standard at the time. None of the big silos supported it, but I wondered how much of it I could implement through their APIs.
A solid amount, as it turned out. Implementing all of the APIs and OStatus would take a lot of work, so I narrowed down the web site part to just WordPress, cranked out an MVP, and launched it on Jan. 8, 2012. Not many other people used it, but I didn’t mind. I wasn’t trying to build a business or a movement, I was just scratching my own itch.
I moved on to other things, but the WordPress part kept nagging at me. I knew I wanted an open standard, and I knew the main candidate was OStatus (and Salmon), but I was reluctant. It was complicated, most web servers didn’t support it, and it was already slowing down in favor of pump.io, the predecessor to ActivityPub.
I searched around a bit, eventually stumbled upon the IndieWeb, and immediately felt at home. I was familiar with Kevin Marks and Tantek, two of the founders, and I was already on board with the principles. When I found webmention and microformats 2, I was hooked. Webmention was barely a year old, but it was exactly the standard I was looking for, without OStatus’s complexity or baggage.
I jumped into the IndieWeb chat, started coding, and quickly added webmention support to Bridgy. I launched the new version on Dec. 9, 2013, and the rest, as they say, is history.
How does it feel, 10 years later? What have I learned? First, as Avery and DHH have described recently, it’s been hugely rewarding to build a tool people want and give it away for free. I built Bridgy primarily for myself; I let other people use it because I believe in sharing, reuse, specialization, and open source. I’d made plenty of other small projects before, but none saw any significant adoption, so I didn’t expect Bridgy to be any different.
It grew slowly and steadily though, through feature launches and silo ups and downs and growth spikes and design debates and plumbing shifts. Today, it has more than 6,000 accounts, sent over 2M webmentions, and processed over 26M silo interactions. It’s even been studied by academic researchers!
It’s also been fun to foster a tiny open source community. If you include the core granary and oauth-dropins libraries, 22 people total have contributed code to Bridgy, and hundreds more have provided bug reports, feature requests, and other feedback. It’s not a big crew – I still maintain the code and production infrastructure largely on my own – but it’s been great to see other people jump in and contribute, often in ways I never would have come up with myself.
Developing Bridgy has been surprisingly helpful career wise, too. It’s where I first learned Python 3, BigQuery, ETL, open source packaging and dependency management, secure coding, unit testing strategies, CI, bits of JavaScript and CSS, and many more things I later used on the job.
More importantly, I’m grateful and proud that Bridgy has helped push the IndieWeb forward a bit. I believe in personal web sites and owning your presence online, so I love that there’s a larger movement building on those ideas. It took a while, but the world is clearly realizing that social networks are a mixed blessing. Now we just have to establish personal web sites as a viable alternative.
Bridgy though. Happy birthday, little buddy! Time passes, things change, silos gonna silo, Bridgy won’t survive forever…but it’s been up and running for 10 years so far, and I have no plans to change that any time soon. Here’s to another 10!
Confession: I didn’t learn either of the two reasons to use a bathroom vent until I was in my mid 20s or so.
…the total outcome of NATs is to increase the available address capacity in IPv4 from 4 billion endpoints (232) to some 1,000 trillion endpoints (250). …the usable address capacity in IPv6 is somewhere between 49 bits and 58 bits. This conclusion points to the observation that the overall carrying capacity of IPv6 is not all that different from that of a dense IPv4 deployment making highly efficient use of NATs.
– Geoff Huston, IPv4 Address Markets (emphasis mine)
Threw in the towel on #AdventOfCode for the season. First time doing it, had a ball, and in silly languages too! Bash, SQL, Emacs Lisp, Prolog. But the volume got to be too much, especially during the holidays. Maybe 1/4 as much would have been perfect. Ah well!