Josh Cunningham wrote a piece called Imagining a Personal Data Pipeline. I started exploring his project, pdpl-cli, which helps you download your personal data and pipe it out to your desired output. I’ve been thinking extensively about this problem for a number of weeks now since I’ve exported my Google Contacts into Obsidian. However, with the lack of database support, I thought about self hosting it. Enter the Personal Data Pipeline.
Overview of the data pipeline
It’s essentially ETL jobs with integrations to third party services to “recipes” that you can write in yaml and customize to your desired outputs. I think this helps a lot more than determining data schemas for specific third party data integrations and having the raw data in a personal data lake. (Or really maybe a document store).
The idea is to have it local-first and maybe include a sync-thing or cloud syncing as an optional add-on. There’s an emphasis on privacy, although my bigger fear is vendor lock-in. I’ve become so reliant on Google, Apple, and other services that I don’t feel like I own my personal data anymore. Also, as a web developer, the hardest part is grabbing my own data from the sticky hands of these cloud services. Also, this emphasis on files over apps makes a lot more sense to me than the walled garden approach we’ve become accustomed to.
Locality of Behaviour is the principle that the behaviour of a unit of code should be as obvious as possible by looking only at that unit of code.
— Carson Gross on talking about htmx
A new version of Ideaverse Pro came out and I spent the weekend configuring it. It’s up now, and I’m getting used to having my setup change. While there’s the obvious changes, like the new theme, there’s more subtle changes, as treating some maps as collections.
SQLite actually has a CSV mode, and in-memory mode. And then it also will take queries directly from the the command line, so you don’t have to go into like little SQLite UI, and do things from there.
React Conf 2024 made me realize that the Apple Event Summary Slides are called “Bento Slides”. I updated the note to reflect that.
My mom used to use a rolodex to keep track of her customers. She owns a hair salon, and she kept a single index card per customer with their name, phone number, and how often they visited. The analog solution made it simple to keep track of these customers, but the biggest pain was retrieving that information.
While I don’t run my own hair salon, I wanted to have my own version of the rolodex. Some people call this a personal CRM, but I never liked the marketing and sales analogy as that’s not the relationship I have with people. In Obsidian, I have a folder called “Rolodex” where each note has a person’s details, like a dossier. Through other notes, I’ll record interactions I’ve had with that person. Not every interaction of course, as that would make the system far too tedious.
A sample of an entry in my rolodex
I imported my contacts over from Google Contacts. I created a script to take a Google sheet export of those contacts and tidy up the information. Of course, I still have to go through and delete contacts who are actually not contacts. And I don’t mind having “weak” contacts as it helps color my notes a lot more when I had a one-time collaboration with them. Also, since Obsidian is bi-directional, I can connect people to each other.
I generally keep a map of contacts in a “map of content” or index of specific groups, like co-workers, activity groups, and close friends. This allows different connections to form for each segment type. Like periodic check-ins. For closer friends, I’ll reach out at least once a year if I have their birthday. Or remember I have to ask when their birthday is.
I created a gist
for it. The original script did not have much error handling. Something to expand
on in the future is to have that error handling within Obsidian, which I would
like to see.
Also, I used the suggester API
to give the user options on which returned item they care about using the Google
Books API.
And here’s my personal template using Obsidian properties.
