Does DuckDB UI support spatial visualizations ? Would be great to be able to use the UI with the spatial extensions.
We support spatial calculations in the UI, but not spatial visualizations just yet. Thanks for the feedback!
Readit News1egg0myegg0 commented on Instant SQL for results as you type in DuckDB UI motherduck.com/blog/intro... · Posted by u/ryguyrg
1egg0myegg0 commented on Instant SQL for results as you type in DuckDB UI motherduck.com/blog/intro... · Posted by u/ryguyrg
This is probably stupid, but at the hope of helping others through exposing my own ignorance -- I'm having trouble actually installing and running the preview... I've downloaded the preview release duckdb binary itself, then when I try to run "duckdb -ui", I'm getting this error:
Extension Autoloading Error: An error occurred while trying to automatically install the required extension 'ui': Failed to download extension "ui" at URL "http://extensions.duckdb.org/0069af20ab/osx_arm64/ui.duckdb_..." (HTTP 403) Extension "ui" is an existing extension.
Is it looking to download the preview version of the extension, but getting blocked/unauthorized (hence the 403 forbidden response)? Or is there something about the auto-loading behavior that I'm supposed to disable maybe?
Sorry you hit that! This is actually already working on version 1.2.2. Could you install that version? That should get you going for the moment! We will dig into what you ran into.
1egg0myegg0 commented on Instant SQL for results as you type in DuckDB UI motherduck.com/blog/intro... · Posted by u/ryguyrg
Will this be available in duckdb -ui ?
Is mother duck editor features available on-prem? My understanding is that mother duck is a data warehouse sass.
It is already available in the local DuckDB UI! Let us know what you think!
-Customer software engineer at MotherDuck
1egg0myegg0 commented on Adding concurrent read/write to DuckDB with Arrow Flight definite.app/blog/duck-ta... · Posted by u/mritchie712
Thank you for all the work you guys do. The Arrow ecosystem is just absolutely incredible.
My few gripes related to interop with duckdb are related to Arrow scanning/pushdowns. And this extends to interop with other projects like pyiceberg too.
Registering an Arrow Dataset (or pyiceberg scan) as a "duckdb relation" (virtual view) is still a little problematic. Querying these "relations" does not always result in an optimal outcome.
For Arrow datasets, you can intercept the duckdb pushdown, but duckdb will have already "optimized" the plan to its liking, and any scanning restrictions that may have been more advantageous based on the nuances of the dataset might have been lost. Eg:
WHERE A IN (3, 5, 7)
Perhaps in a similar way, turning an pyiceberg scan into a relation for duckdb effectively takes the entire scan and creates an Arrow Table rather than some kind of pushdown/"scan plan" for duckdb to potentially make more efficient with its READ_PARQUET() functionality.
Most of this is probably dependent on duckdb development, but all of the incredible interop work done across communities/ecosystems so far gives me a lot of confidence that these will soon be matters of the past.
IN list filter predicate pushdown is much improved in DuckDB 1.2, coming in about a week! I am not sure if it applies to Arrow yet or not. Disclaimer: I work at MotherDuck and DuckDB Labs
1egg0myegg0 commented on Databases in 2024: A Year in Review cs.cmu.edu/~pavlo/blog/20... · Posted by u/avinassh
Every time I've tried to use DuckDB it has segfaulted on me, so yeah I'm betting on DataFusion..
That is very surprising to hear! If you can reproduce it, could you please file a bug report on GitHub? That would be a huge help!!
1egg0myegg0 commented on Tubeworms live around deep-sea vents economist.com/science-and... · Posted by u/thm
Are we in the Dune timeline?
1egg0myegg0 commented on DuckDB Community Extensions duckdb.org/2024/07/05/com... · Posted by u/isaacbrodsky
Honest question, how feasible it would be for DuckDB to release a non-columnar version of their DB (or at least make DuckDB a decent choice for a typical web app)? I don't know any other DB that makes installing extensions this easy. The rate at which they're shipping awesome features makes me wonder if they could eventually become a great generic database.
I know, I know, this could just as easily be a double-edged sword. A database should prioritize stability above everything else, but there is no reason why we shouldn't expect them to reach there.
Hello! I would recommend trying out DuckDB's SQLite attach feature! You can read or write data, and even make schema changes, all with DuckDB's engine and syntax. The storage then uses SQLite, which is row oriented!
https://duckdb.org/docs/extensions/sqlite
(I work at MotherDuck and DuckDB Labs)
1egg0myegg0 commented on DuckDB Community Extensions duckdb.org/2024/07/05/com... · Posted by u/isaacbrodsky
> What happens behind the scenes is that DuckDB downloads an extension binary
The baser part of me wonders how hard it would be to compromise that supply chain.
Extension downloads are validated using a signature check to prevent tampering!
(I work for DuckDB Labs and MotherDuck)
1egg0myegg0 commented on What Happens When You Put a Database in the Browser? motherduck.com/blog/olap-... · Posted by u/self
Um, my data lake is measured in Pb, not Tb. How is that going to work exactly?
Howdy! I work at MotherDuck and DuckDB Labs (part time as a blogger). At MotherDuck, we have both client side and server side compute! So the initial reduction from PB/TB to GB/MB can happen server side, and the results can be sliced and diced at top speed in your browser!
Have they fixed the incredibly slow queries on indexed columns?
Howdy! Thanks for your benchmarking!
Your blog does a great job contrasting the two use cases. I don't think too much has changed on your main use case, however here are a few ideas to test out!
DuckDB can read SQLite files now! So if you like DuckDB syntax or query optimization, but want to use the SQLite format / indexes, that may work well.
Since DuckDB is columnar (and compressed), it frequently needs to read a big chunk of rows (~100K) just to get 1 row out and decompressed. Mind trying to store your data uncompressed? Might help in your case! (PRAGMA force_compression='uncompressed')