Rendered at 16:06:23 GMT+0000 (Coordinated Universal Time) with Cloudflare Workers.
dofm 21 minutes ago [-]
Just add a line in AGENTS.md that says "never talk about Minecraft unless you're explicitly asked", I'm sure it'll be fine after that.
Tiberium 2 hours ago [-]
Sounds like a hallucination unless proven otherwise, even the leading LLMs can do those from time to time, and they will always appear plausible like that. Also could be the session having a lot previous context, like 800K+, which (I think) makes hallucinations more likely.
Relevant comment from the OP which makes a hallucination more likely:
> There is one tool call result that includes a string that printed a pathname including minecraft.py because it was listing the files in a Python virtual environment and the Pygments package has a lexer called minecraft.py
andy99 25 minutes ago [-]
I realize hallucination has no precise definition but this doesn’t sound at all like anything I’ve ever heard called hallucination. Hallucination is usually plausible wrong answers or made up info that ends up fitting the most likely response (like a manufactured citation) and comes from the way LLMs work at predicting tokens. This example demonstrates completely implausible output, it’s not something that fits with hallucination.
All that said, it doesn’t require cross session leakage, it could just be training data or like those nightingale (probably the wrong bird*) data generations where they just prompt an LLM with nothing and it starts spitting out conversations.
I see a bunch of downstream comments about caching, sounds like maybe there’s an error where it loads nothing instead of the cache and so starts spitting out random generations.
* edit: it’s magpie. Worth looking at the concept, I’m not sure people realize they LLMs generate random conversations when prompted with nothing, this seems at least as likely as sessions leaking: https://github.com/magpie-align/magpie
macNchz 1 hours ago [-]
The person posting this claims to have reproduced in a separate context down the thread:
> Same thing just happened on a Claude Mobile session in same Enterprise account. Common theme in both is Sonnet 5, first response after more than 5 minutes (cache miss).
xyzzy_plugh 2 hours ago [-]
I don't disagree but this sort of thing has to be investigated regardless.
It's unfortunate that there is so little transparency that even if they deny there was a leak we will never know for certain.
alserio 42 minutes ago [-]
Why? what does make it more likely?
bix6 49 minutes ago [-]
So the options are this amazing tech is so stupid it just randomly brings up Minecraft or it’s got a major security issue?
27183 47 minutes ago [-]
¿Por qué no los dos?
andy99 5 minutes ago [-]
Interesting to see the claudeslop reply as the first comment to the gh post and the reaction to it.
Note the repro condition: first response after 5+ min, i.e. a cache miss. A cache leak would show up on hits (someone else's cached prefix), not on misses where everything is recomputed from your own tokens.
acepl 2 hours ago [-]
Oh yes, we do not need programmers any more…
JohnMakin 19 minutes ago [-]
it’s the wet dream of execs and pm types. however, i have not seen anything close to it in my life. I remember the UML days, lol. the issue is not the code, it’s the translation layer between business and code. maybe someday ai bridges that gap. history has shown probably not
emehex 2 hours ago [-]
"Coding is largely solved"
supriyo-biswas 29 minutes ago [-]
The funny thing is at my current employer, they mentioned that "coding is increasingly becoming a solved problem" and in the same breath, mentioned that one project was too hard for anyone to do so they're not doing it and would rather sell existing features...
consp 1 hours ago [-]
While abused by LLM vendors, that phrase in one form or another I've been hearing since the early '00s and it's likely way older.
ethagnawl 59 minutes ago [-]
Sure but have you ever seen it actually play out in practice like it currently is? Whether or not it's true (of course it's not) people are currently behaving as if it is and firing/hiring accordingly.
philipov 30 minutes ago [-]
Well, when was the last time you wrote machine code by hand?
... but then they went and changed what coding meant.
We've always been layering abstractions on top of abstractions. If we get to an abstraction that works well enough that you no longer have to dive down into the previous layer, we say we've solved coding, and change what coding means. Obviously LLMs aren't there yet.
techpression 1 hours ago [-]
I love that quote, especially considering the insane amount of bugs that are produced.
It’s as easy to debunk as someone claiming ”I can jump to the moon”.
kylehotchkiss 1 hours ago [-]
50% unemployment :D
ec109685 1 hours ago [-]
Caching doesn’t work the way the bug reporter implies. Caches are shared (at least across the enterprise), but its key is always a function of the input before it.
We achieved significant savings simply by moving everything that varies across individuals out of the system prompt so every session starts from a cache point.
For example you never want your system prompt to start with the time that the session started. Move that to the first user message if needed.
macNchz 1 hours ago [-]
Caching is not supposed to work like that, but that doesn’t preclude the cache key computation function from having bugs.
marginalia_nu 1 hours ago [-]
Yeah there's quite a lot of potential bugs that could have this shape. If I were to guess it could be a buffer in a buffer pool not being sized and zeroed correctly, allowing stale data to bleed between sessions.
estebarb 31 minutes ago [-]
Hash functions necesarily have collisions. Also, it is perfectly possible to introduce bugs in the hash function (hash inputs, hash function itself) that allows cross account contamination.
Waterluvian 42 minutes ago [-]
There is a massive incentive for optimization, so I expect they’re doing a ton of very clever tricks, all of which make this kind of bug more likely.
supriyo-biswas 1 hours ago [-]
There could just also be a bug where the output tokens of session 1 were shared with session 2, due to a race condition or similar.
ai_fry_ur_brain 14 minutes ago [-]
Openrouters model providers give me urls people have given them quite frequently.
44 minutes ago [-]
bfeynman 11 minutes ago [-]
fwiw, this could be a bug but the submitters level of arrogance places this rather high on the dunning-kruger side of things. There are multiple other plausible explanations, but this person is probably vibe coder who believes anything an llm says (including explaining its own hallucinations)
jstummbillig 1 hours ago [-]
Is there anything particular about LLMs that would make separating customer data harder than in all SaaS cases?
adam_arthur 40 minutes ago [-]
Vibe-coding the implementation.
I haven't had much issue with Codex, but seems Claude Code has major issues being reported nearly on the daily.
They also happen to be the most boastful about not reading or looking at the code.
LLMs are very capable, but not nearly to the level they seem to be messaging.
(We've actually moved on from vibe-coding to having the LLM vibe code itself in a loop)
27183 31 minutes ago [-]
> having the LLM vibe code itself in a loop
The businesslatin name for this is Recursive Self-Improvement
rabbidruster 28 minutes ago [-]
Interestingly I had an almost identical experience to this report in codex. It output a user memory file that looked awfully real and wasn't at all related to my work.
27183 58 minutes ago [-]
If I had to hazard a guess, doing anything in a multi-tenant way on a GPU is going to be hard mode compared to most SaaS due to lack of memory safe tooling. I've built multi-tenant SaaS systems, and I've done a little GPU programming (a long time ago), but I've never tried to combine the two disciplines.
woadwarrior01 54 minutes ago [-]
It'd be terribly compute inefficient to not share prefix caches (KV cache) across customers.
acepl 45 minutes ago [-]
What is the probability that two customers will have exactly the same tokens in cache? Wouldnt it require using the exact same CLAUDE.md, skills, MCPs and context? After that it is even worse since the nondeterminism of LLMs and humans
27183 39 minutes ago [-]
I suspect what GP is getting at is there will be a strong incentive to implement some structural sharing across tenants to avoid redundantly storing the same tokens over and over. At least I'd be tempted to do this if I was working with a very precious, constrained resource (e.g. VRAM). Doing this correctly seems.. very difficult. [edit] To answer your question directly: the probability that the entire cache is identical between two different users is very low, but the probability that there exists identical chunks of cache between two different users is very high. Exploiting those commonalities successfully will significantly compress the data.
dezgeg 11 minutes ago [-]
System prompt for something like Claude Code should be identical, no?
57 minutes ago [-]
TZubiri 14 minutes ago [-]
0 evidence. If this were a real privacy leak, the author would ask their coworker if they talked about the unexpected topic instead of
>"Maybe my coworker was talking about this in another session?"
This would be a critical bug that would slash the market value of a T$ company significantly, go ask your coworker or close the ticket, why do you expect the devs to put an enormous amount of effort hunting a potentially inexistent if you can't make that minuscule debugging effort.
Kapura 43 minutes ago [-]
happy fourth of july everybody!
ofjcihen 15 minutes ago [-]
Happy fourth to you too :)
Avicebron 1 hours ago [-]
In order Fable 5 has rejected:
"Recipe for red-braised pork, I have pork shoulder"
"Write up a framework for MCP patterns I can give to claude code"
"explain the biomechanics of motion in c. elegans" (I get this one, I mostly did it to test and it's related to my hobby project)
Do we get an extra day of functional Fable 5 because it's down?
andy99 22 minutes ago [-]
Not sure the relevance of this comment, but normally if someone built a classifier that bad they’d be fired. Anthropic obviously thinks they have some monopoly power they can use to foist garbage on consumers, I think they don’t.
nijave 15 minutes ago [-]
The safety filter rejected or the model was down?
HumanOstrich 25 minutes ago [-]
What does this have to do with anything? Who are you talking to? This is Hacker News, not Anthropic support.
asveikau 11 minutes ago [-]
HN becoming anthropic support would certainly explain a lot of threads and comments I've seen here lately. Thank you for this.
Relevant comment from the OP which makes a hallucination more likely:
> There is one tool call result that includes a string that printed a pathname including minecraft.py because it was listing the files in a Python virtual environment and the Pygments package has a lexer called minecraft.py
All that said, it doesn’t require cross session leakage, it could just be training data or like those nightingale (probably the wrong bird*) data generations where they just prompt an LLM with nothing and it starts spitting out conversations.
I see a bunch of downstream comments about caching, sounds like maybe there’s an error where it loads nothing instead of the cache and so starts spitting out random generations.
* edit: it’s magpie. Worth looking at the concept, I’m not sure people realize they LLMs generate random conversations when prompted with nothing, this seems at least as likely as sessions leaking: https://github.com/magpie-align/magpie
> Same thing just happened on a Claude Mobile session in same Enterprise account. Common theme in both is Sonnet 5, first response after more than 5 minutes (cache miss).
It's unfortunate that there is so little transparency that even if they deny there was a leak we will never know for certain.
... but then they went and changed what coding meant.
We've always been layering abstractions on top of abstractions. If we get to an abstraction that works well enough that you no longer have to dive down into the previous layer, we say we've solved coding, and change what coding means. Obviously LLMs aren't there yet.
We achieved significant savings simply by moving everything that varies across individuals out of the system prompt so every session starts from a cache point.
For example you never want your system prompt to start with the time that the session started. Move that to the first user message if needed.
I haven't had much issue with Codex, but seems Claude Code has major issues being reported nearly on the daily.
They also happen to be the most boastful about not reading or looking at the code.
LLMs are very capable, but not nearly to the level they seem to be messaging.
(We've actually moved on from vibe-coding to having the LLM vibe code itself in a loop)
The businesslatin name for this is Recursive Self-Improvement
>"Maybe my coworker was talking about this in another session?"
This would be a critical bug that would slash the market value of a T$ company significantly, go ask your coworker or close the ticket, why do you expect the devs to put an enormous amount of effort hunting a potentially inexistent if you can't make that minuscule debugging effort.
"Recipe for red-braised pork, I have pork shoulder"
"Write up a framework for MCP patterns I can give to claude code"
"explain the biomechanics of motion in c. elegans" (I get this one, I mostly did it to test and it's related to my hobby project)
Do we get an extra day of functional Fable 5 because it's down?