jc4p (u/jc4p) - Readit News

jc4p commented on Show HN: SwiftAI – open-source library to easily build LLM features on iOS/macOS github.com/mi12labs/Swift... · Posted by u/mi12-root

mi12-root · 6 hours ago

That’s a good suggestion, and it indeed sounds like something we’d want to support. Could you help us better understand your use case? For example, where do you usually get the models (e.g., Hugging Face)? Do you fine-tune them? Do you mostly care about LLMs (since you only mentioned llama.cpp)?

jc4p · 4 hours ago

Thank you! I’ve been fine tuning tiny Llama and Gemma models using transformers then exporting from the safetensors that spits out — My main use case is LLMs but I’ve also tried getting YOLO finetuned and other PyTorch models running and ran into similar problems, just seemed very confusing to figure out how to properly use the phone for this.

jc4p commented on Show HN: SwiftAI – open-source library to easily build LLM features on iOS/macOS github.com/mi12labs/Swift... · Posted by u/mi12-root

jc4p · 6 hours ago

I do a lot of AI work and right now the story for doing LLMs on iOS is very painful (but doing Whisper or etc is pretty nice) so this is existing and the API looks Swift native and great, I can't wait to use it!

Question/feature request: Is it possible to bring my own CoreML models over and use them? I honestly end up bundling llama.cpp and doing gguf right now because I can't figure out the setup for using CoreML models, would love for all of that to be abstracted away for me :)

jc4p commented on Tea app leak worsens with second database exposing user chats bleepingcomputer.com/news... · Posted by u/akyuu

shkkmo · a month ago

Doesn't that Gemini summary gist tie usernames to pretty specific highly personal non-public stories? That seems like a significant violation of ethical hacking principles.

jc4p · a month ago

They're anonymous usernames the app had them make and they were told don't use anything shared elsewhere and I googled and there's not any uniquely identifiable people from any of them.

They seem generic enough that I think it's okay, but you're right there is no need in including them and I should've caught that in the AI output, thank you!!

jc4p commented on Tea app leak worsens with second database exposing user chats bleepingcomputer.com/news... · Posted by u/akyuu

coopreme · a month ago

Are you concerned about potential CFAA issues?

jc4p · a month ago

Yes! haha! But hopefully I have a good enough support group and connections that I'll be ok if that happens, I just really wanted to prove that they were not being honest when they said it was data prior to 2024.

jc4p commented on Tea app leak worsens with second database exposing user chats bleepingcomputer.com/news... · Posted by u/akyuu

jc4p · a month ago

Hi all, i'm the security researcher mentioned in the article -- just to be clear:

1. The leak Friday was from firebase's file storage service

2. This one is about their firebase database service also being open (up until Saturday morning)

The tl;dr is:

1. App signed up using Firebase Auth

2. App traded Firebase Auth token to API for API token

3. API talked to Firebase DB

The issue is you could just take the Firebase Auth key, talk to Firebase directly, and they had the read/write/update/delete permissions open to all users so it opened up an IDOR exploit.

I pulled the data Friday night to have evidence to prove the information wasn't old like the previous leak and immediately reached out to 404media.

Here is a gist of Gemini 2.5 Pro summarizing 10k random posts: https://gist.github.com/jc4p/7c8ce9a7392f2cbc227f9c6a4096111...

And to be 100% clear, the data in this second "leak" is a 300MB JSON file that (hopefully) only exists on my computer, but I did see evidence that other people were communicating with the Firebase database directly.

If anyone is interested in the how: I signed up against Firebase Auth using a dummy email and password, retrieved an idToken, sent it into the script generated by this Claude convo: https://claude.ai/share/2c53838d-4d11-466b-8617-eae1a1e84f56

And here's the output of that script (any db that has <100 rows is something another "hacker" wrote to and deleted from): https://gist.github.com/jc4p/bc35138a120715b92a1925f54a9d8bb...

jc4p commented on Normalizing Flows Are Capable Generative Models machinelearning.apple.com... · Posted by u/danboarder

godelski · 2 months ago

You have a few minor errors and I hope I can help out.

  > Diffusion: generate a lot of noise then try to clean it up

You could say this about Flows too. The history of them is shared with diffusion and goes back to the Whitening Transform. Flows work by a coordinate transform so we have an isomorphism where diffusion works through, for easier understanding, a hierarchical mixture of gaussians. Which is a lossy process (more confusing when we get into latent diffusion models, which are the primary type used). The goal of a Normalizing Flow is to turn your sampling distribution, which you don't have an explicit representation of, into a probability distribution (typically Normal Noise/Gaussian). So in effect, there are a lot of similarities here. I'd highly suggest learning about Flows if you want to better understand Diffusion Models.

  > The diffusion approach that is the baseline for sota is Flow Matching from Meta

To be clear, Flow Matching is a Normalizing Flow. Specifically, it is a Continuous and Conditional Normalizing Flow. If you want to get into the nitty gritty, Ricky has a really good tutorial on the stuff[0]

[0] https://arxiv.org/abs/2412.06264

jc4p · 2 months ago

thank you so much!!! i should’ve put that final sentence in my post!

jc4p commented on Normalizing Flows Are Capable Generative Models machinelearning.apple.com... · Posted by u/danboarder

jc4p · 2 months ago

i've been trying to keep up with this field (image generation) so here's quick notes I took:

Claude's Summary: "Normalizing flows aren't dead, they just needed modern techniques"

My Summary: "Transformers aren't just for text"

1. SOTA model for likelihood on ImageNet 64×64, first ever sub 3.2 (Bits Per Dimension) prev was 2.99 by a hybrid diffusion model

2. Autoregressive (transformers) approach, right now diffusion is the most popular in this space (it's much faster but a diff approach)

tl;dr of autoregressive vs diffusion (there's also other approaches)

Autoregression: step based, generate a little then more then more

Diffusion: generate a lot of noise then try to clean it up

The diffusion approach that is the baseline for sota is Flow Matching from Meta: https://arxiv.org/abs/2210.02747 -- lots of fun reading material if you throw both of these into an LLM and ask it to summarize the approaches!

jc4p commented on Ask HN: Conversational AI to Learn a Language · Posted by u/edweis

sampleuser58 · 3 months ago

One modification I would suggest is to add a bit more to the initial prompt like:

"write as if you are a person from {{REGION}}. Modify your language to proficiency level {{PROFICIENCY_LEVEL}}"

that way I could for example, speak as if it's someone using Mexican Spanish vs Madrid Spanish vs Chilean Spanish, etc.

Secondly, you could include the user's speech transcribed as part of the conversation window

jc4p · 3 months ago

Amazing idea, do you think this should be a freeform text field the user can enter to add their own prompts to or should it be a checkbox/select on the homepage so the user can pick from a limited set?

jc4p commented on Ask HN: Conversational AI to Learn a Language · Posted by u/edweis

jeffwass · 3 months ago

Did you just add Dutch as per the submitter’s request or was it part of your plan prior?

Curious because I’m trying to learn Romanian, and since it’s a less common language there are fewer resources available. So I wasn’t sure if you added Dutch with minimal amount of effort following the poster’s request.

That said, I gave your app a try with Spanish and it looks pretty good! But I didn’t see a Help page to clarify how I’m “supposed” to interact. Eg I tried saying in English “I don’t understand” (even though I know how to say that in Spanish) and it responded in Spanish which may be hard for absolute beginners. Although full immersion is much better way to learn.

I can try playing around more with it to give you some feedback.

jc4p · 3 months ago

I just added Romanian for you -- here's the entire diff for adding a new language (as long as it's in OpenAI's training data) -- https://images.kasra.codes/romanian_diff.png

Please let me know if it works, and I'll definitely work on adding in instructions for the expected interactivity, thank you!

jc4p commented on Ask HN: Conversational AI to Learn a Language · Posted by u/edweis

valleyer · 3 months ago

This is great! Well done.

I've used the realtime API for something similar (also related to practicing speaking, though not for foreign languages). I just wanted to comment that the realtime API will definitely give you the user's transcriptions -- they come back as an `server.conversation.item.input_audio_transcription.completed` event. I use it in my app for exactly that purpose.

jc4p · 3 months ago

Thank you so much!! While the transcription is technically in the API it's not a native part of the model and runs through Whisper separately, in my testing with it I often end up with a transcription that's a different language than what the user is speaking and the current API has no way to force a language on the internal Whisper call.

If the language is correct, a lot of the times the exact text isn't 100% accurate, if that's 100% accurate, it comes in slower than the audio output and not in real time. All in all not what I would consider feature ready to release in my app.

What I've been thinking about is switching to a full audio in --> transcribe --> send to LLM --> TTS pipeline, in which case I would be able to show the exact input to the model, but that's way more work than just one single OpenAI API call.