This week I meant to focus on researching and shaping up a project for Contacts and finishing some of the small tasks I still had in Linear. In addition some more tasks popped up during the week.
Similar to the last week, the tasks during this week had very trivial fixes which I've spent up to a day to figure out 🤦♂️.
Deleting documents
There was a problem when a document with multiple generations was deleted — it was appearing in the list, but GetDocument for it would fail.
So, turns out the DB query assembling the list of documents wasn't right — it had a problem in the GROUP BY clause, which was filtering the deleted documents before they are grouped, but what we actually wanted is to select the last generation for each document first, and then filter out the deleted ones. So the fix was quite trivial (in addition to the test I wrote to debug it).
Collaborators not being able to move or delete documents
This was really weird. It was reported that collaborators of a site are not able to move or delete documents. I was able to reproduce this locally, and I started researching. I couldn't find any obvious problems, so I wrote a test for it. The test was very weird and sometimes flaky.
And then after spending some more time on that, it just started working consistently. I think I pulled some code in the middle, but it didn't seem related at all.
Then I made a few tests manually and it all seemed to work fine. ¯\_(ツ)_/¯
Arbitrary signatures API
This is a bit controversial, and I don't really like the fact that we need this, but I didn't want to debate it. So now we are able to ask the daemon to sign arbitrary data with any account key that it has.
Network protocols switching issue
We used to have only one Libp2p protocol ID: /hypermedia/<version>. Last week we've introduced another one that we use for linking multiple devices into one account: /hypermedia/devicelink/<version>.
When two peers connect we first validate if they are able to speak the same protocol, and the process for doing that was relying on the protocol ID prefix being /hypermedia. So now, because two valid protocols have this prefix, sometimes we were taking the devicelink one and then would determine that the peer can't speak our protocol.
This caused problems with syncing and showing the peer list in the app, which would randomly and intermittently show some peers as invalid.
I unified the approach for protocol verification, and instead of error-prone manual string manipulations I've added a simple regex match as a first step of the verification:
Contacts Project
I haven't finished the Contacts project yet, and I haven't fully written it up. I was researching the way AT Proto approached it with their newly added identity verification feature. I have mixed feelings about that for our system.
I think in our system we should start the simpler Petname/Edgename contact system, which has some overlapping concerns with identity verification system like Bluesky's, but I think would be more useful in our, more decentralized, system.
This system would need mutability and the ability to delete data, which overlaps with the need to remove documents, comments, and members. For now we've only implemented deleting documents, but we've done it in a way that still leaves some data behind (the tombstone Ref blob).
I think we need to spend some time thinking on whether there's a more scalable and unified approach for deleting data that we could use for our system.
In general deleting data in a decentralized system is pretty hard, but maybe at least we could have a bit more principled approach in terms of how do we delete the various data types that we manage.