thebuilderjr (u/thebuilderjr)

thebuilderjr commented on Apache DataFusion datafusion.apache.org/... · Posted by u/thebuilderjr

bionhoward · 8 months ago

How does this compare/contrast to polars? Seems pretty similar, anybody tried both?

thebuilderjr · 8 months ago

DataFusion and Polars are like two sides of the same Rust coin: DataFusion is built for distributed, SQL-based analytics at scale, serving as the backbone for data systems and enabling complex query execution across clusters. Polars, on the other hand, is laser-focused on blazing-fast, single-node data manipulation, offering a Python-like DataFrame API that feels intuitive for exploratory analysis and in-memory processing.

thebuilderjr commented on Apache DataFusion datafusion.apache.org/... · Posted by u/thebuilderjr

krapht · 8 months ago

I feel like I'm not the target audience for this. When I have large data, then I directly write SQL queries and run them against the database. It's impossible to improve performance when you have to go out to the DB anyway; might as well have it run the query too. Certainly the server ops and db admins have loads more money to spend on making the DB fast compared with my anti-virus laden corporate laptop.

When I have small data that fits on my laptop, Pandas is good enough.

Maybe 10% of the time I have stuff that's annoyingly slow to run with Pandas; then I might choose a different library, but needing this is rare. Even then, of that 10% you can solve 9% of that by dropping down to numpy and picking a better algorithm...

thebuilderjr · 8 months ago

I agree. The main reason I shared it is because I find it interesting as a library. I actually use it behind the scenes to build https://telemetry.sh. Essentially, I ingest JSON, infer a Parquet schema, store the data in S3 with a lookaside cache on disk, and then use DataFusion for querying.

thebuilderjr commented on Show HN: Turn Your Study Materials into Interactive Quizzes with AI syncstudy.app... · Posted by u/bukunmiadewale3

Mizu_048 · 8 months ago

hey, i quite like it but answering a quiz and then it generating an explaination is kind of distracting, i suggest a mode for it to create a quiz with collectively 10-20 questions, after which it shows which were correct and which were false and only generates an explaination when asked or when we click some explain button

again great site!!!

thebuilderjr · 8 months ago

That's a great idea. I think there are pros/cons to both approaches. By having the AI adapt every question, in theory it would be more responsive and personalized. On the other hand it's a more clunky UI compared to traditional quizzes. Perhaps the key is to have "modes" where users can opt in depending on the type of experience they are looking for. I'll see what I can do :)

thebuilderjr commented on Show HN: Turn Your Study Materials into Interactive Quizzes with AI syncstudy.app... · Posted by u/bukunmiadewale3

thebuilderjr · 8 months ago

Great minds think alike :P I literally just launched https://quizchat.ai yesterday on ShowHN haha

thebuilderjr commented on Show HN: QuizChat – Learn new things using LLM powered quizzes quizchat.ai... · Posted by u/thebuilderjr

thebuilderjr · 8 months ago

Hi HN,

I recently built a fun little Christmas project called QuizChat. It’s like ChatGPT but designed to help you learn and refresh your knowledge on various topics. The idea came to me while working on some ML/PyTorch projects, where I realized my tensor reasoning skills could use some improvement. I wished for a tool like this to help me practice and get better.

What sets QuizChat apart is the chat interface, which shines when you get a question wrong. Unlike traditional flashcard or quiz platforms, you can use the freeform textbox to ask for tips, tricks, or clarifications, making the learning experience much more interactive and engaging.

I’d love to hear your thoughts! Feel free to leave a comment below or email me at hi@quizchat.ai.

Thanks!