Readit News logoReadit News
ozgune commented on Mistral OCR 3   mistral.ai/news/mistral-o... · Posted by u/pember
Tiberium · 2 days ago
From a tweet: https://x.com/i/status/2001821298109120856

> can someone help folks at Mistral find more weak baselines to add here? since they can't stomach comparing with SoTA....

> (in case y'all wanna fix it: Chandra, dots.ocr, olmOCR, MinerU, Monkey OCR, and PaddleOCR are a good start)

ozgune · 2 days ago
Also, do you know if their benchmarks are available?

In their website, the benchmarks say “Multilingual (Chinese), Multilingual (East-asian), Multilingual (Eastern europe), Multilingual (English), Multilingual (Western europe), Forms, Handwritten, etc.” However, there’s no reference to the benchmark data.

ozgune commented on Pg_lake: Postgres with Iceberg and data lake access   github.com/Snowflake-Labs... · Posted by u/plaur782
ozgune · 2 months ago
This is huge!

When people ask me what’s missing in the Postgres market, I used to tell them “open source Snowflake.”

Crunchy’s Postgres extension is by far the most ahead solution in the market.

Huge congrats to Snowflake and the Crunchy team on open sourcing this.

ozgune commented on Benchmarking Postgres 17 vs. 18   planetscale.com/blog/benc... · Posted by u/bddicken
anarazel · 2 months ago
Afaict nothing in this benchmark will actually use AIO in 18. As of 18 there is aio reads for seq scans, bitmap scans, vacuum, and a few other utility commands. But the queries being run should normally be planned as index range scans. We're hoping to the the work for using AIO for index scans into 19, but it could work end up in 20, it's nontrivial.

It's also worth noting that the default for data checksums has changed, with some overhead due to that.

ozgune · 2 months ago
If the benchmark doesn’t use AIO, why the performance difference between PG 17 and 18 in the blog post (sync, worker, and io_uring)?

Is it because remote storage in the cloud always introduces some variance & the benchmark just picks that up?

For reference, anarazel had a presentation at pgconf.eu yesterday about AIO. anarazel mentioned that remote cloud storage always introduced variance making the benchmark results hard to interpret. His solution was to introduce synthetic latency on local NVMes for benchmarks.

ozgune commented on DeepSeek OCR   github.com/deepseek-ai/De... · Posted by u/pierre
yoran · 2 months ago
How does an LLM approach to OCR compare to say Azure AI Document Intelligence (https://learn.microsoft.com/en-us/azure/ai-services/document...) or Google's Vision API (https://cloud.google.com/vision?hl=en)?
ozgune · 2 months ago
OmniAI has a benchmark that companies LLMs to cloud OCR services.

https://getomni.ai/blog/ocr-benchmark (Feb 2025)

Please note that LLMs progressed at a rapid pace since Feb. We see much better results with the Qwen3-VL family, particularly Qwen3-VL-235B-A22B-Instruct for our use-case.

ozgune commented on AI Coding: A Sober Review   ubicloud.com/blog/ai-codi... · Posted by u/furkansahin
softwaredoug · 3 months ago
This space is filled with personal anecdotes and studies from providers. It's hard to get objective perspectives from independent labs.
ozgune · 3 months ago
(Disclaimer: Ozgun from Ubicloud)

I agree with you. I feel the challenge is that using AI coding tools is still an art, and not a science. That's why we see many qualitative studies that sometimes conflict with each other.

In this case, we found the following interesting. That's why we nudged Shikhar to blog about his experience and put a disclaimer at the top.

* Our codebase is in Ruby and follows a design pattern uncommon industry * We don't have a horse in this game * I haven't seen an evaluation that evaluates coding tools in (a) coding, (b) testing, and (c) debugging dimension

ozgune commented on Deploying DeepSeek on 96 H100 GPUs   lmsys.org/blog/2025-05-05... · Posted by u/GabrielBianconi
ozgune · 4 months ago
The SGLang Team has a follow-up blog post that talks about DeepSeek inference performance on GB200 NVL72: https://lmsys.org/blog/2025-06-16-gb200-part-1/

Just in case you have $3-4M lying around somewhere for some high quality inference. :)

SGLang quotes a 2.5-3.4x speedup as compared to the H100s. They also note that more optimizations are coming, but they haven't yet published a part 2 on the blog post.

u/ozgune

KarmaCake day1634February 21, 2011
About
Ubicloud, Microsoft, Citus Data, Amazon
View Original