True, I understand why this is better than all those companies scraping it individually (for both ability to opt out and site load), but the way they handled opt out is still quite silly.

They also scraped some of my assembly code of which I’m pretty sure the latest version has a major bug. Let’s hope they don’t use these models to program my new PC’s bios.

The right move is to private all repositories you are uncertain of that you want them to be preserved forever in an AI training set.

I just followed the opt out link and they’re making you open a public issue with a Markdown list with all your repositories you want removed.

Surely there has to be a better way to do this (probably the point).

Also why are they storing 4.71 TB in a Git repo? How are they going to deal with removal requests?

EDIT: It seems like they’re manually responding to the issues, wtf? There’s a perfectly fine GitHub auth system that they could use to verify everyone’s GitHub account / repository ownership

EDIT 2: This is apperently a collaboration between Hugging Face and ServiceNow, why is my companies IT ticketing system scraping all of GitHub?

Roko’s basilisk

My list includes GPL licensed repo’s

It doesn’t seem that way. My list contains several repositories that don’t have licenses, even some with GPL.

No, it seems to only be a subset of public repo’s.
I have like 65 repo’s and only 13 were scraped. I don’t get why they specifically scraped those though. They don’t have the most stars, they aren’t the oldest or newest, not the ones with the most forks, nor do I see a pattern based on programming language.

Too many people gloss over the fact how insane it is that some people genuinely believe non-believers burn forever in a magic post death torture world. Like how can someone think that while talking with someone, and be like, aw sucks for them, glad I submit to this ancient chain email creepypasta.

It still baffles me when I realize someone thinks like that.

I don’t think the cost of hosting the repositories itself is the issue, more the legal problems and the fact that real projects get drowned out by the noise

It’s interesting how certain companies and organizations have such large ranges, 16m IP’s each for both that old printer company and a farmaceutical company is a lot. It really shows the history of the internet and how seemingly certain companies that adopted it first ended up with huge chunks of the available IPv4 space.

Better than that

Haven’t used Claude code myself, so I wouldn’t know, but a commit to delete only a comment is indeed pretty weird, also most of the commit messages are the GitHub default like “Update gui.h” which is also a bit odd.

Looking at the GitHub repo it seems like the first commit was actually just 2 weeks ago and contained 12k lines. I can’t spot any AGENT.md files, but it does feel like the author quite new to this. That could be explained by them being 16 though like they say on their profile.

Deepseek recently published a paper in which they describe that vision tokens contain more information than text tokens and that this can be used to compress context.

We present DeepSeek-OCR as an initial investigation into the feasibility of compressing long contexts via optical 2D mapping.

Experiments show that when the number of text tokens is within 10 times that of vision tokens (i.e., a compression ratio < 10×), the model can achieve decoding (OCR) precision of 97%. Even at a compression ratio of 20×, the OCR accuracy still remains at about 60%. This shows considerable promise for research areas such as historical long-context compression and memory forgetting mechanisms in LLMs.

It reminds me of LLM caveman speak, it used to have another option to use Chinese instead of English. A language like Chinese is seemingly better at encoding information in fewer tokens and I think this is the same mechanism why OCR tokens work so well.

That said, I also doubt that voice messages are more efficient than text prompts, but it’s best not to waste too much time engaging with these sorts of LinkedIn posts (and LinkedIn in general).

This does not seem convenient at all, from the bash script to get relative links, to the file tags which are not synced across devices by standard git or file sync.

Great choice, using some of that VC money for things that actually benefit the world

I think the original post/image kinda was, but the response/retweet doesn’t seem like it

Fun fact the LessWrong guy Yudkowsky who was/is an influential person in silicon valley wrote one of the most well known Harry Potter fanfics. At some point he apparently tried to get JK Rowling’s permission to publish it.

Go seems like a good option to begin with; you can do a lot with it, and it’s not that complicated but does expose you to concepts like pointers.

There are plenty of very “serious” systems written in Go (e.g. Kubernetes), it’s not a toy language.