Cleaning up my PDS data - mnlwldr

Over the last couple of days, I've been experimenting with various standard.site applications. My goal was simply to get a better understanding of the available tools and figure out which ones fit best into my workflow.

This post, however, isn't about the tools themselves. Instead, it's about a small issue I ran into: Offprint left data behind in my PDS that should have been deleted.

The problem

During my testing, I created a few test publications in Offprint and later deleted them. At least, that was the idea.

Inside Offprint, the publications were gone. However, other applications and services consuming the same ATProto data continued to display them. Since the publications no longer existed in Offprint, I had no way to delete them from there a second time.

So I decided to take a look directly at my PDS.

Inspecting the PDS

To browse my repository, I used the ATProto Browser at https://www.atproto-browser.dev. It provides a convenient way to inspect the collections and records stored in your Personal Data Server.

It didn't take long to find the culprit. Under the app.offprint.publication collection, the test publications were still present, even though they should already have been deleted.

One of the things I really like about ATProto is how transparent everything is. Application data ultimately lives as records inside your own repository. Applications like Offprint are just clients that create and consume those records. As long as you have the appropriate credentials, you can manage them directly through the standardised XRPC API, regardless of whether the application itself exposes that functionality.

Since the browser currently doesn't support deleting records, I quickly put together a small Go application.

Deleting records through the API

The application authenticates using my handle and an app password, lists all records in the app.offprint.publication collection using com.atproto.repo.listRecords, and then deletes them one by one using com.atproto.repo.deleteRecord.

The code certainly isn't pretty, but it did exactly what I needed for this one-off cleanup.

type Record struct {
	URI string `json:"uri"`
}

type ListResponse struct {
	Records []Record `json:"records"`
}

func main() {

	// fetch token with handle and app password
	tokenResponse, err := getToken(handle, password)
	if err != nil {
		log.Fatal(err)
	}

	// request records from the collection
	req, _ := http.NewRequest(
		"GET",
		fmt.Sprintf("%s/xrpc/com.atproto.repo.listRecords?repo=%s&collection=%s",
			pds,
			tokenResponse.DID,
			collection,
		),
		nil,
	)
	req.Header.Set("Authorization", "Bearer "+tokenResponse.AccessJwt)

	resp, err := http.DefaultClient.Do(req)
	if err != nil {
		panic(err)
	}
	defer resp.Body.Close()

	var list ListResponse
	if err := json.NewDecoder(resp.Body).Decode(&list); err != nil {
		panic(err)
	}
	fmt.Printf("Found %d records\n", len(list.Records))

	for _, rec := range list.Records {
		fmt.Printf("found record=%s\n", rec.URI)

		parts := bytes.Split([]byte(rec.URI), []byte("/"))
		rkey := string(parts[len(parts)-1])

		body, _ := json.Marshal(map[string]string{
			"repo":       tokenResponse.DID,
			"collection": collection,
			"rkey":       rkey,
		})

		req, _ := http.NewRequest(
			"POST",
			pds+"/xrpc/com.atproto.repo.deleteRecord",
			bytes.NewReader(body),
		)

		req.Header.Set("Authorization", "Bearer "+tokenResponse.AccessJwt)
		req.Header.Set("Content-Type", "application/json")

		resp, err := http.DefaultClient.Do(req)
		if err != nil {
			fmt.Println("Error:", err)
			continue
		}

		io.Copy(io.Discard, resp.Body)
		resp.Body.Close()

		fmt.Println("Deleted", rkey)
	}
}

After running the program, the app.offprint.publication collection was clean. Unfortunately, the test publications were still showing up in several places.

I took another look through my repository and eventually found another collection: site.standard.publication.

The same test publications were stored there too.

Since that collection also contained records I wanted to keep, deleting everything wasn't an option. Instead, I removed only the specific records by calling the same API endpoint with the corresponding record URIs.

Only then did the test publications finally disappear from every application.

My assumption is that creating a publication in Offprint also creates a corresponding Standard publication record. While one of those records was removed, the other remained behind. Whether this is an actual bug in Offprint or simply something caused by my testing, I can't say for certain.

A word of caution

Working directly with your PDS is incredibly powerful, but it should also be done carefully. Deleting records through the XRPC API means you're interacting directly with your repository instead of going through an application's user interface. If you accidentally delete the wrong record, restoring it is usually not possible unless you have a backup.

Conclusion

Cleaning up these test records wasn't just a way to get rid of some leftover data—it was also a great opportunity to better understand the structure of my own PDS.

Moments like this are a big part of what makes ATProto so interesting to me. It's refreshing to know that my data isn't trapped inside a black box. Instead, I can inspect it, understand it, and, when necessary, manage it myself.