jbylund (u/jbylund) - Readit News

jbylund commented on Python-based compiler achieves orders-of-magnitude speedups news.mit.edu/2023/codon-p... · Posted by u/Stratoscope

benj111 · 2 years ago

They should have said python compiler (shorter) or python compiler in c++ (more accurate and only one character longer, including spaces).

Considering at least 2 people have gone to look at the source and then come here to comment, it would have been a net benefit for all involved. Plus, what does it say about the potential quality of your compiler if you can't even make correct English statements? This seems easier to get right than if( x = *p++ )

jbylund · 2 years ago

These headlines usually work by the dept asking a researcher for a 10 sentence summary of their work. Someone in the dept summarizes that to 3 sentences, and sends that to the university pr dept. The university turns that 3 sentences into 1 and that's what's published. My take is this says more about the weird game of telephone being played than it does about the research product itself.

jbylund commented on Pypi.org is running a survey on the state of Python packaging pypi.org/... · Posted by u/zbentley

mdmglr · 3 years ago

My wishlist:

We need a way to configure an ordered list of indexes pip searches for packages. —extra-index-url or using a proxy index is not the solution.

Also namespaces and not based on a domain. So for example: pip install apache:parquet

Also some logic either in the pip client or index server to minimize typosquatting

Also pip should adopt a lock file similar to npm/yarn. Instead of requirements.txt

And also “pip list” should output a dependency tree like “npm list”

I should not have to compile source when I install. Every package should have wheels available for the most common arch+OS combos.

Also we need a way to download only what you need. Why does installing scipy or numpy install more dependencies than the conda version? For example pywin and scipy.

jbylund · 3 years ago

Typosquatting is a thing that has been looked at and is being looked at:

https://github.com/pypi/warehouse/pull/5001 - had to be reverted because it was too noisy

https://github.com/pypi/warehouse/issues/9527

jbylund commented on Pypi.org is running a survey on the state of Python packaging pypi.org/... · Posted by u/zbentley

LukeShu · 3 years ago

If the package is available as a wheel, you don't need to execute code to see what the requirements are; you just need to parse the "METADATA" file. However, the only way to get the METADATA for a wheel (using PyPA standard APIs, anyway) is to download the whole wheel.

For comparison, pacman (the Arch Linux package manager) packages have fairly similar ".PKGINFO" file in them; but in order to support resolving dependencies without downloading the packages, the server's repository index includes not just a listing of the (name, version) tuple for each package, it also includes each package's full .PKGINFO.

Enhancing the PyPA "Simple repository API" to allow fetching the METADATA independently of the wheel would be a relatively simple enhancement that would make a big difference.

----

As I was writing this comment, I discovered that PyPA did this; they adopted PEP 658 in March of this year! https://github.com/pypa/packaging.python.org/commit/1ebb57b7...

jbylund · 3 years ago

Pip can use range requests to fetch just a part of the wheel, and lift the metadata out of that. So it can sometimes avoid downloading the entire wheel just to get the deps. Some package servers don't support this though.

Also, there's a difference between a pep being adopted and that pep being implemented (usually a bunch of elbow grease). That said there are a couple exciting steps towards 658 being implemented: https://github.com/pypa/pip/pull/11111 (just approved yesterday, not yet merged) https://github.com/pypi/warehouse/issues/8254 (been open for forever, but there has been incremental progress made. Warehouse seems to not attract the same amount of contribution as pip)

jbylund commented on PostgreSQL views and materialized views and how they influenced TimescaleDB timescale.com/blog/how-po... · Posted by u/od0

jbylund · 3 years ago

Did you consider a type of continuous aggregate that works over pre-aggregated partitions for time buckets without modifications and swapping those buckets in the continuous aggregate view with a live view when data is modified? I guess it would mean that performance would sort of continually degrade as inserts/updates/deletes happened in more and more time buckets, but as soon as the aggregation event fired performance would recover. It seems like one could provide a more strongly consistent view of aggregates with this sort of approach.

jbylund commented on Things I can’t do on macOS which I can do on Ubuntu (2020) shkspr.mobi/blog/2020/04/... · Posted by u/robin_reala

mikeryan · 3 years ago

1) inability to ask the window manager "which workspace/desktop am I on" - it also seems like this ability has been progressively removed with newer versions (judging from stackoverflow/askdifferent questions) - after some hacking I've found a workaround for this

I just swipe up with three fingers on the Trackpad to see what desktop I'm on, are you looking for something different?

jbylund · 3 years ago

Yeah, wanted to be able to programmatically ask the window manager such that I can tell new shells to source "~/.bashrc.d/{desktopname}.sh".

jbylund commented on Things I can’t do on macOS which I can do on Ubuntu (2020) shkspr.mobi/blog/2020/04/... · Posted by u/robin_reala

boringg · 3 years ago

Am I the only one who actually finds more desktop a drawback then a benefit? I clearly haven't mastered multiple desktops -- do you split between projects and just leave it there always? I haven't figured out how to best optimize that which I feel I am leaving real value on the table.

jbylund · 3 years ago

I have a browser profile per workspace and git/aws/maybe some other env vars that are set up on a per workspace basis. To switching from project 1 to project 2 or personal I just click on the appropriate desktops in the status bar.

Which reminds me, another missing thing is the ability to have desktops displayed in the status bar and click on the one you want to jump to.

jbylund commented on Things I can’t do on macOS which I can do on Ubuntu (2020) shkspr.mobi/blog/2020/04/... · Posted by u/robin_reala

jbylund · 3 years ago

As a new mac user (~4 weeks with new employer) my biggest N are:

1) inability to ask the window manager "which workspace/desktop am I on" - it also seems like this ability has been progressively removed with newer versions (judging from stackoverflow/askdifferent questions) - after some hacking I've found a workaround for this

2) clicking on a browser notification just brings me to _any_ window of the browser, not the one that sent the notification! - if anyone knows how to fix this please help me out

3) there's a delay on toggling caps lock that you can't disable without installing some small third party utility! it just feels super rude that they're essentially telling me that I didn't mean to press caps lock

jbylund commented on Function pipelines: Building functional programming into PostgreSQL blog.timescale.com/blog/f... · Posted by u/oconnor663

jbylund · 4 years ago

What's the explain plan for a pipeline like this look like?

In my experience one of the biggest things scaring devs away from moving more logic over the wire towards the db is lubrication between version control and database state. If I end up defining 10s or 100s of custom pipeline operators in git how do I keep that in sync with the db? and am I responsible for doing all that legwork myself?