Why - Motivation
Most data scientists and researchers I follow live in timezones that are quite far from mine. That makes it very easy for me to miss valuable tweets. It is also hard to read past a hundred tweets in the timeline. And finally I want to keep easily accessible lists of interesting tweets. This projects aims to solve these problem.
@Smerity did a better job explaining the problem in his tweets:
I hate to say but @Twitter sucks for knowledge. Amidst the flame wars and trolls, there is genuinely valuable discussions had here - but how the hell do you find it. You can't order tweets by popularity. Search barely works. I have a hard enough time finding my own tweets.
— Smerity (@Smerity) June 15, 2018
Hilariously the reason this is likely an issue is that using @Twitter for knowledge is not the standard use case. Most tweets are intended to be transient. Most tweets are never worth more than a cursory look. Yet there are stunning evergreen threads that are lost forever.
— Smerity (@Smerity) June 15, 2018
How - Methodology
The gist of the process:
- Collect tweets on my timeline periodically and automatically.
- Cluster collected tweets according to their topics and sort them by popularity.
- Read relevant tweets and picked the ones I find interesting enough.
- Convert picked tweets and combine them into a web page.
The tweets are split by day. A day starts at 08:00 UTC, which was selected based on the pattern of activities on my timeline:
The time used to put a tweets in a day is the time it appeared on my timeline, not the creation time of the tweet (only retweets are affected).
Contributions Welcome!
Here are some ways to contribute:
- Recommend people to follow.
- Improve the posts. (Everything is open-sourced.)
- Suggest additional tweets to a day (through Github pull requests or issues).
- Any general suggestions to the project (through Github issues.)