Readit News logoReadit News
tomohawk · 7 years ago
One of the biggest benefits of go vs python is that you only manage your dependencies during build. You end up with a single executable that is easy to deploy and mostly immune to the deployment of other things. In python, you have to also worry about deployment. To be safe, you need to carry the vm and other dependencies with you.

This means you can't just give any python program to someone and have it just work, unless it's pretty trivial.

sametmax · 7 years ago
Not entirely true anymore.

Nuitka (nuitka.net) has been able to compile python programs to a single stand alone executable reliably, robustly and easily for a while now.

It's a real gem and deserves to be more popular. We are very far from the quirks of pyinstaller or cxfreeze : it just works, even with numpy or qt.

Yes, go is still superior for deployment :

- it can cross compile

- executables are smaller

- it's built in so it's just an easier experience

- it doesn't depend on libc

But it's important to spread the word: the time were your python program was hard to ship is over.

Just use nuitka.

amerine · 7 years ago
op literally said:

> To be safe, you need to carry the vm and other dependencies with you.

nine_k · 7 years ago
Go also carries all its dependencies with it; this is why it produces large single static binaries.

Python lacks a tool to painlessly produce the same, that is, a self-contained binary that includes the VM and all the dependencies. Determining Python dependencies may be slightly tricky, because `import` is an executable statement, and dependencies may be native-code libraries.

There are several tools for that, but all have their downsides, and none is in widespread use. Polishing such a tool until it's as simple and fool-proof as Go's build tool could be quite beneficial for Python community.

mickael-kerjean · 7 years ago
Everyone seem to agree the generation of a static binary is easy with go. That's not my experience as soon as CGO and the glib is involved. I would gladly pay 100$ to make it work on my open source project using libvips to anyone who can create the static build: https://github.com/mickael-kerjean/nuage
ofrzeta · 7 years ago
Why does it have to be a self-contained binary? When you pack everything into an virtualenv you get more or less the same result, don't you? If you want to have a single file you can zip it :)
_ph_ · 7 years ago
Right, its not magic to build such a tool. But until it exists, this is a benefit for Go. This is also why I keep using Tcl, with tclkit you can build single-file executables for you tcl programs.
eivarv · 7 years ago
Pyinstaller solves the problem for me, but it can (in some particular cases) be a real bitch to configure properly with your dependencies at first.
sametmax · 7 years ago
Such a tool now exists and is called nuitka. See my other comment. It's very mature and can be trusted even with compiled extensions.
linuxftw · 7 years ago
Speaking from the linux distro perspective, this is not true.

You can develop an application in python that utilizes distro-supplied python libraries. If you package your library in .deb or .rpm, it's trivial to utilize the dependencies from the distro.

Otherwise, you can install from pypi via pip, and especially into a virtualenv. You can declare each dependency's specific version if you so choose. Modern distros support python2.7 and python 3.x.

If you're on some other operating system, yeah, using python sucks for you.

While it's fair to say python is not statically compiled, it's no different than managing dependencies in other dynamically linked software.

Unlike python, go applications have to be cross compiled. For the most part, native python code will run anywhere the interpreter runs.

In the world of containerized applications, vendoring dependencies is a moot point anyway. We're all shipping around big tarballs with everything baked in anyway, so go doesn't offer any advantages in that space other than smaller binary size.

Not to say that go doesn't have some advantages over python, but I think the dependency management problem is vastly overstated, at least when it comes to python.

Spivak · 7 years ago
> You can develop an application in python that utilizes distro-supplied python libraries.

And unless you're aggressively tracking your distro's package releases you'd better hope that the new libdep doesn't introduce any breaking bugs. Also your app is no longer for Linux it's for Ubuntu 18.04 updated roughly on 2018-08-03.

The same goes for the actual python. Distributions apply lots of downstream patches and backport fixes so your python 2.7 is really redhat-python2.7-13 so you might want to test against that -- or you can bundle your own and be done with it.

Unless you have the wall clock time to actually define and test supported distributions you probably want to pretend the system python doesn't exist.

> pypi via pip, and especially into a virtualenv

I would never subject my users to this workflow. What you're describing are source distribution channels which are (ab)used to distribute applications. I have no idea if some dependency's native extension will even compile on their system. Do they even have build tools? Why would they?

Spiritus · 7 years ago
Unless your application also contains other assets like images etc. Now you suddenly need to package and install/deploy your Go application.
deadbunny · 7 years ago
Or you just bundle them all in to the binary like a monster.
takeda · 7 years ago
And TBH properly packaging Go program is a nightmare if you want packager to build it and the build machine has no access to net.
deadbunny · 7 years ago
Statically linked binaries are hardly new. As with everything they come with tradeoffs.
shaan7 · 7 years ago
Depends on your usecase, really. Static linking has been there for a long time and it has its disadvantages. I have more than a dozen programs on my system that share the same Python runtime and libraries. Compare that to every Go program shipping its own copy of the same things.
mrweasel · 7 years ago
>every Go program shipping its own copy of the same things.

Is that really an issue? It takes up a bit more space, but nothing that's really any concern on a modern system. People often resort to using Docker to manage dependencies in other programming languages and end up shipping an entire copy of an operating system.

I honestly don't see many downside to shipping a static binary. At least in that case you can chroot the entire thing, something I really wish Python (and most other languages) would make a whole lot easier.

Security fixes require a recompilation, that perhaps the largest downside, but if you use Docker, then you need to rebuild that image as well.

emmelaich · 7 years ago
Absolutely true.

You should make it easy as possible for users of your software .

akvadrako · 7 years ago
Of course you can do the same with python - that's what docker is for.
KaiserPro · 7 years ago
First let me state that a language is just a brush with which to paint your logic. I really don't subscribe to the X is better than Y. I might concede that x is better than y for certain tasks, but not a sweeping generalisation.

The first point about inconsistent spelling mistakes, I 100% sympathise. I suffer from this terribly. Pylint and syntastic are a silver bullet for me. I have been tempted by a more fancy IDE, but I'm so used to vim, I'm reluctant/scared to change.

Parallelism: I do look over the wall at Go and wish that python 3.x had spent more time on removing the GIL, but that is old news now.

However I enjoy threading with python(not its limitations), because its so trivially easy. The Queues primitive saves 90% of the headache. as for ctrl-c not doing anything set all your threads to daemon (<threadObj>.setDaemon(True)) When the main thread exits, so does everything else. Go shines by having built in true parallelism

If I need horsepower, then I have to run multiple processes (not multiprocessing, that has drawbacks.) which has the annoyance of serializing and message passing. (but that does mean scaling horizontally less painful further on). Again, Go really shines here.

Statically compiling thing: YES. This means that I can 90% of the faff with docker (xyz version not working any more, the build fails) and use a real scheduling system that runs on real steel(or AWS) and not the omnifaff that is kubernetes.

codemusings · 7 years ago
> I have been tempted by a more fancy IDE, but I'm so used to vim, I'm reluctant/scared to change.

I'm happy to report that JetBrains' VIM plugin is almost flawless (Block-based editing, window splitting, save/close, it's all where it's supposed to be).

I used to be in the same camp. But if you add a linter and parser and full blown backend to make lookups for code completion to your VIM configuration you might as well use an IDE. I've switched to IntelliJ for Java.

abhishekjha · 7 years ago
And they provide a free all pack subscription for open source contributors. Its nice to see all the features at one place though their pipenv integration is still a bit shaky[0]. I haven't got a response yet.

[0] : https://youtrack.jetbrains.com/issue/PY-30855

KaiserPro · 7 years ago
I did see that it had vim support, but its always difficult to know which _bits_ of vim it supports. Thanks for this, I'll investigate.
earenndil · 7 years ago
I think the pypy people were able to get a trivial program running with the gil unlocked. I'd watch them for further updates.
fifnir · 7 years ago
Regarding multiprocessing and serializing between processes in python, and linux specifically, have you tried SharredArray?

It dumps numpy arrays in /dev/shm which can loaded by other processes by the filepath, muuuuuch nicer than passing things around

KaiserPro · 7 years ago
Ooo, I have not. This sounds interesting.
JanisL · 7 years ago
I think when you really need performance it's a great example of a time to start looking at alternative Python implementations, CPython just isn't the best choice for a lot of tasks that actually have processing speed requirements.
BerislavLopac · 7 years ago
For starters, just adding Cython and/or Numba to the code should introduce great improvements. https://rushter.com/blog/numba-cython-python-optimization/
kqr · 7 years ago
> I really don't subscribe to the X is better than Y. I might concede that x is better than y for certain tasks, but not a sweeping generalisation.

Not even if X is better than Y for every specific task?

KaiserPro · 7 years ago
_if_ it was, then I'd be using that brush instead of the other. but to be honest, most languages are pretty much the same
k_bx · 7 years ago
For those who lack the generics, I just wanted to remind that you can "script" in Haskell by adding this to your header:

    #!/usr/bin/env stack
    -- stack script --resolver=lts-12.2
and run it as an executable if your machine has stack installed. If not -- compile it with a `stack ghc --resolver=lts-12.2 ./file.hs` and you'll have a `./file` executable. No project creation, no need to list the dependencies.

exceptione · 7 years ago
Good trick! Do you also know the Turtle library? I just got across it, and it looks like a neat shell library from the outside.
gsibble · 7 years ago
I've written in Python for ~13 years and I'm really enjoying Go. It's fast and its libraries (the few there are) seem to make a lot of sense. I don't think I would use it for daily scripting like the author, but for a large new project I would definitely pick Go over Python now.
lmm · 7 years ago
Try an ML-family language (e.g. OCaml, F#, or Rust) if you haven't already. You get the best of all worlds: expressiveness, safety and speed.
doteka · 7 years ago
I like the expressiveness of Rust in theory, but for the programs I write I don't feel like I get sufficient payoff for fighting the borrow checker, as compared to writing in Go. Would I be correct in saying OCaml is pretty much Rust without the borrow checker? I have no prior experience with the language.
claydavisss · 7 years ago
Why can't he just be happy using Go?
arendtio · 7 years ago
Does someone know a ressource where I can easily learn how to switch from Bash to Go?

To give some context: When it comes to serious programs I use Go already, but often I just want to code a little script to watch a website for changes for example:

  #!/usr/bin/env bash
  url="https://example.com"
  mem="memory.txt"
  oldsum="$(cat "$mem")"
  newsum="$(curl -sL "$url" | sha512sum)"

  if [ "$oldsum" != "$newsum" ]; then
    xmpp_notify "$url changed"
    echo "$newsum" > "$mem"
  fi
I haven't tried it yet, but in my mind it feels like writing such a program would be more time consuming/complex in Go than writing it in bash. Neverthelss, I feel like Go is overall a much better language than Bash (with all those quotes), so I would like to transition to writing more short programs in Go and for that I would like to learn how to do things like reading/writing/grepping files as easily in Go as I do it in Bash. Any idea if someone wrote some hints together already?

chomp · 7 years ago
The benefit of bash is that it has system binaries as first class citizens; you can leverage the hundreds of thousands of lines of GNU projects with a single keyword. It's hard to beat that expressiveness.

Saying "Go is overall a much better language" is like saying a wrench is a better tool than a screwdriver. You should keep both in your toolbelt and switch out when the need arises.

For your question, start with the regexp and os package, learn how to open a file, read in the contents, and then while iterating over the contents, apply a regex to the text. That should get you to the point that you can search plaintext for character strings. Good luck.

IshKebab · 7 years ago
Writing a program like that would not be too difficult in Go. It's something like this:

    func main() {
      url := "https://example.com"
      mem := "memory.txt"
      oldsum, err := ioutil.ReadFile(mem)
      if err != nil {
        log.Fatal(err)
      }
      
      // Download URL
      resp, err := http.Get(url)
      if err != nil {
        log.Fatal(err)
      }
      defer resp.Body.Close()
      body, err := ioutil.ReadAll(resp.Body)
      if err != nil {
        log.Fatal(err)
      }
      
      // Hash it
      newsum := sha512.Sum512(body)
      
      if oldsum != newsum {
        err = exec.Command("xmpp_notify", url + " changed").Run()
        if err != nil {
          log.Warn(err)
        }
        err = ioutil.Write(mem, newsum)
        if err != nil {
          log.Error(err)
        }
      }
    }
Don't listen to the naysayers. Go is fantastic for scripts like this - it's really good at network stuff, has a ton of built in functions, etc. And you get actual robustness and error checking that isn't a complete joke. Look how many ways your Bash script could fail! That's fine if you're literally watching the output with your eyes. If not it's going to cause issues when `curl` fails for example.

The only downside really is that you have to compile the Go code, which is fast but not as fast as just interpreting a Bash script (the first time).

Another possible issue is executable size - if you have a lot of Go scripts they may take up a fair bit of space.

I wish Go had an interpreter / JIT mode for scripts.

Edit: Before someone says "but that's 3 times longer than the bash code!" please write the equivalent Bash code that has proper error checking.

boris · 7 years ago
Just add this as a second line?

  trap "exit 1" ERR

arendtio · 7 years ago
Thanks for giving an equivalent example in Go :-)

I think for this small size of an example program the error checking in Go is pure overhead, but as soon as the program starts to have a few more functions it will be very useful to have more options than just to abort on fail (Bash trap style / set -e). So I don't have any problem with 'polluting' my source with `if err != nil` checks ;-)

What I am more concerned about are the different ways I have to call functions from different libraries like

  resp, err := http.Get(url)
  ...
  defer resp.Body.Close()
  body, err := ioutil.ReadAll(resp.Body)
That is a lot more 'special' than the other function calls and the bash version and probably requires more documentation reading than '$ curl --help'. But hey, thanks to your example I have a starting point and probably are going to experiment with it in the new future.

For all the others who wrote that Bash is the right tool for the job: I read your warnings, opinions and suggestions and will keep them in mind while exploring the possibilities of using Go for script like jobs.

h1d · 7 years ago
That is so repetitive to handle errors. Doesn't go have a cleaner way of handling errors than making the code look far less readable without unified error handling like exceptions?
ainar-g · 7 years ago
As a professional gopher, don't.

Just a few months ago I replaced an almost 100-line Go "script" with two lines in the Makefile. The fact that Go is great for big projects doesn't automatically make it good for everything. Go is for programs, not scripts.

arendtio · 7 years ago
Thanks for your warning :-)

My problems come from the other end: My Bash scripts tend to become larger. A few weeks ago I wrote a 'cache' utility function (very useful to cache curl requests for example). A few days ago I wanted to use it in a second script was just about to write a package manager for bash modules... I think that is the kind of situation you are getting into when the line count of a bash script increases and you are trying to do proper software development ;-)

So I am not going to build every script in Go from today on, but I would like to learn a bit more on when Go is a viable alternative to Bash.

watshn · 7 years ago
Being such a concise language, Haskell could be a good option here. You can do quick scripting while still having a powerful type system (especially compared to Go's) backing you up. I somewhat quickly typed this code up and haven't tested it, but it should replicate that bash script pretty closely:

  #!/usr/bin/env stack
  -- stack script --resolver=lts-12.2
  
  import System.Process (system)
  
  import Control.Lens
  import Crypto.Hash.SHA256
  import qualified Data.Bytestring.Lazy as B
  import Network.Wreq
  
  url = "https://example.com"
  mem = "memory.txt"
  
  main = do
      oldsum <- B.readFile mem
      newsum <- (hashLazy . view responseBody) <$> get url
      when (oldsum /== newsum) $ do
          exitCode <- system $ "xmpp_notify " ++ url ++ " changed"
          B.writeFile mem newsum
I didn't make use of it, but there's also the Turtle library which aims to provide a fairly solid shell scripting experience, providing a lot of coreutils as simple Haskell functions. You also get a REPL in the form of GHCI, which comes default in Haskell installations.

dilap · 7 years ago
I love Go and use it for allllmost everything, but I think bash is still better for something like this. The ease with which it can run programs and capture output is the killer feature.

I'd love to see some future bash like language that's strongly typed, has a consistent and simple syntax, but retains these advantages.

baq · 7 years ago
i'd say bash is the perfect tool for the problem that you demonstrated. go may be a better language but it's just another tool.
IshKebab · 7 years ago
Depends how much you care about robustness. If the answer is "more than zero" than Bash is a terrible tool.
rezeroed · 7 years ago
Go is the wrong tool for that. I'm no fan of bash, but it's often the the quick, clear (enough) solution for small tasks. I'm not a python dev - maybe that would be a suitable compromise?
lalaithion · 7 years ago
for me, I use bash for anything less than 10 lines long, python for anything between 10 and 500 lines, and then I switch to a heavier programming language for anything longer.
swaroop · 7 years ago
Or you could use Python with https://pypi.org/project/plumbum/ :)
greyman · 7 years ago
> I’m not a fan of full blown IDEs that require their own configuration.

Hmm, but those IDEs actually do solve most of the problems author has had with Python. If one regularly code in Python, spending a few hours configuring the IDE is justifiable. I for example never write scripts outside the IDE, because why?

kungtotte · 7 years ago
Note: I like Python, so this is me playing devil's advocate here.

If you need an IDE to solve common pain points of a language, isn't it fair to say that another language which lacks those pain points is a better fit for you? Using an IDE to aid you, to smooth things out and allow for more rapid development and better workflow is one thing. Being forced to use one because the language is lacking is a bad look on the language!

jacobush · 7 years ago
You could also say that this attitude holds us back. What if Smalltalk had only been accessible from vi back in the day, would Smalltalk have been as appreciated? I think not.
IshKebab · 7 years ago
No they don't. No IDE can detect all typos in Python and do accurate code completion like they can for statically typed languages because the problem is unsolvable in general.

They can use heuristics to get fairly okish. Pycharm is not bad and VSCode's new Python extension seems to work fairly well. But it's never going to be as rock solid as Java code completion for example.

heavenlyblue · 7 years ago
It seems the author is one of those employees who think if he works hard he does a good job.
SjuulJanssen · 7 years ago
Some issues:

- I make stupid mistakes in Python constantly. I misname a variable or a function or I pass in the wrong arguments: Try mypy

- If the task is IO bound: Threads are the wrong the answer to that. You're going async by using Golang, why not in python?

- With Python, I have to ensure all the packages I need are installed on the remote machine: pipenv Like slezyr mentions

- Consistent Styling: Try autopep8 or black

- Intellisense: I would say "use Pycharm" but vscode is getting better I heard

raverbashing · 7 years ago
Pylint is a bit hard to configure, use Prospector

Prospector does a better job and it prevents you from wasting time with what essentially are stylistic choices and BS like a hard 80 chars limit per line (remember, A Foolish Consistency is the Hobgoblin of Little Minds)

And yes, yes, automated stilling tools are better

Intelisense is needed but even the standard vim autocompletion plugin does an ok job

cakoose · 7 years ago
> You're going async by using Golang, why not in python?

In Golang, you're still using a thread-based abstraction.

In Python, "going async" involves switching to a slightly different style of programming and using different I/O libraries. While that has advantages, using threads lets you stick with the standard style.

StavrosK · 7 years ago
I had a very good experience using PyInstaller for a few of my Python programs, it made "binaries" out of them really easily.
vortico · 7 years ago
This is like saying "Goodbye car, hello airports!" If your needs change, then of course your favorite solution may change.