Readit News logoReadit News
didip · 5 years ago
I am surprised that it doesn't have public interfaces with context.

Network file system could use the same interfaces + timeout settings.

infogulch · 5 years ago
Oh god no, this draft is actually pretty good, don't ruin it by infecting it with context. Context is the worst library in Go: it's a pile of hacks to get cancellation and goroutine-local storage by manually passing new heap-allocated objects through every function interface in every library between low-level IO up to task management. It infects everything, obliterating separation of concerns making even libraries that should have nothing at all do do with timeouts include code for it. And it includes an untyped key-value store implemented as a linked-list of pairs (!!!!), because why not?

If you can't tell, I don't like context. I've said before [0] I really hope that Go 2 comes up with an actual solution to the cancellation and task-local storage problems and deprecates context. Some comments in that thread pointed to alternatives that looked pretty decent, I wonder what state they're in these days.

[0]: https://news.ycombinator.com/item?id=18561884

zeveb · 5 years ago
I quite like the concept of contexts, I just wish they were implicit rather than explicit. The idea of a function taking place in a dynamic context makes perfect sense (because, after all, it does); the only thing which doesn't really make sense is having to pass it along by hand. Imagine how terrible it would be if we had to explicitly pass the return stack everywhere[0]!

It needs to 'infect' (as you put it) everything because it needs to pervade everything, so a library can pass it through to anything it calls. This is a good argument for making it implicit.

> And it includes an untyped key-value store implemented as a linked-list of pairs (!!!!), because why not?

That is just a cons chain, famous for example as the foundation of the Lisp programming language, and there is nothing wrong with it. Note that it is unfair to say that it is untyped: the values themselves are strongly typed, and the cells too are typed — interface{} (or T in Lisp terms) is still a type!

A cons chain has advantages for inheritance of values in a DAG of calls. It is not necessarily the most efficient, but simplicity is a virtue.

0: Continuation-passing style is both really powerful and well-nigh unreadable, for a reason.

thiht · 5 years ago
I love contexts. They're a simple solution to a complicated problem. They take up space in the functions signatures but they're sooo easy to read and to use.

I really hope they'll stay.

networkimprov · 5 years ago
No Go 2.0 release is planned. Go2 is a marketing label (I guess?) meant to imply that the Go 1.x language and stdlib are still evolving.

The term is a mistake, as it creates the impression that a backwards-incompatible product is in the works.

solatic · 5 years ago
Everything old is new again. This challenge is basically why monads exist, because the monad allows you to cleanly separate between the state in which the algorithm is running and the algorithm itself. Reminds me of: https://philipnilsson.github.io/Badness10k/posts/2017-05-07-...

I wonder what poor abstraction the Go authors will come up with instead?

icholy · 5 years ago
There was discussion about this. One approach would be to tie the entire fs.FS instance to a context. Something akin to `http.Request.WithContext`.
jeffbee · 5 years ago
How would that be helpful? A context deadline is a fixed point in time. You really want the context (and its deadline) to be specified at the operation scope, not at the file or at the entire filesystem level. And you might have a deadline on a read but be willing to wait forever for a close, so you don't really want the deadline to be attached to the file handle, either.
joshuak · 5 years ago
I am not a fan of the context pattern either, however one can support it without need to explicitly define that support.

    var fs fs.Fs
    fs = somefs.New()
    fs = contextfs.New(context.TODO(), fs)
fs is now an fs with an imbedded context.

nikon · 5 years ago
Accept interfaces, return structs?
suessflorian · 5 years ago
Awesome to see some work on filesystem API's but oh boy... you can really see the challenge provided by the backwards compatibility promise. Alias's in os package to definitions in io/fs, the duplication of the http.FileServer.
shawnz · 5 years ago
Much of the criticism I have seen towards Go has been in regard to the poor design of its filesystem APIs (example: https://news.ycombinator.com/item?id=22443363). So rethinking this might make the language significantly more attractive for some. Especially considering the addition of generics, I am becoming much more interested in the language again.

Deleted Comment

SEJeff · 5 years ago
I’m kind of surprised they don’t mention afero at all, who was one of the first big packages to abstract filesystem access. Shoutout as they make unit testing filesystem ops a piece of cake!

https://github.com/spf13/afero

kyrra · 5 years ago
They are likely aware of it, as spf13 works on golang at Google. :)
SEJeff · 5 years ago
Yeah I realize that. It is just surprising is all. It is a very nice to use interface. Getting an equivalent in stdlib would be great.
mseepgood · 5 years ago
SEJeff · 5 years ago
Awesome find, thanks!
shanemhansen · 5 years ago
Just a random gopher sharing their level of surprise. I didn't know about the spf13 project, but I instantly thought of https://godoc.org/golang.org/x/tools/godoc/vfs

Looks like they mention that package in their proposal.

LTClipp · 5 years ago
I would like to put it out there that I'm working on a pathlib library that is attempting to solve a lot of the problems that this design draft is addressing.

https://github.com/chigopher/pathlib

networkimprov · 5 years ago
This (recent but buried) comment suggests replacing FileInfo in the ReadDirFS interface with a DirEntry type, for reasons of performance and future extensibility:

https://www.reddit.com/r/golang/comments/hv976o/qa_iofs_draf...

It was prompted by this proposal for os.Readdirentries():

https://github.com/golang/go/issues/40352

It seems to me that FileInfo isn't a suitable interface for either a general filesystem construct, or a performant implementation.

bfuclusion · 5 years ago
Is stat really something you want as an abstraction over all filesystems? You might not even have that information available in all of them.
skissane · 5 years ago
Every filesystem ever has the concept of file attributes.

The problem is the exact details of what file attributes are supported vary widely from system to system.

The best approach is to support a subset which is reasonably common across platforms – file type[1], modification timestamps, file size in bytes, etc – and an extension mechanism to enable platform-specific attributes.

Java NIO handles this reasonably well with the java.nio.file.attribute package[2] in my opinion. (Not sure how easy it would be to port the concepts of that to Go though.)

IANA has a registry of OS-specific facts (i.e. file attributes) and OS-specific file types[3] – this is for use of FTP MLST and MLSD commands[4] but the registry is rather empty because that RFC doesn't appear to have got much adoption. It is a good idea though.

[1] There is a standard list of file types most platforms support – regular file, directory, link – but there are lots of special file types specific to various platforms (e.g. named pipes, UNIX domain sockets, BSD whiteouts, NTFS junctions), plus some filesystems have different subtypes of regular files or directories. (For example, on IBM z/OS, a "regular file" could be a UNIX file, a VSAM dataset, or a non-VSAM dataset, and the later two both have several subtypes; similarly, z/OS has UNIX directories, but PDS(E) could also be viewed as a non-UNIX type of directory.)

[2] https://docs.oracle.com/en/java/javase/14/docs/api/java.base...

[3] https://www.iana.org/assignments/os-specific-parameters/os-s...

[4] https://tools.ietf.org/html/rfc3659

heinrichhartman · 5 years ago
Calling stat on all the files is just really expensive if your directly is large.
networkimprov · 5 years ago
See the Reddit for an alternative to FileInfo.

https://www.reddit.com/r/golang/comments/hv976o/qa_iofs_draf...

tptacek · 5 years ago
FileInfo is an interface, isn't it? You could populate fields other than "name" lazily, right?
sanxiyn · 5 years ago
This is very valuable for languages to get right. Tcl had it for a long time and I envy them: https://wiki.tcl-lang.org/page/VFS