Goroutines are, for all practical purposes, threads. They're cheaper than real threads but they still have all the subtle complexity that comes with shared-memory preemptive threading (everything shared needs to be appropriately locked, scheduling is non-deterministic and can happen any time, etc). Async/await is different; context switches can only happen at an `await` expression. This greatly reduces the number of possibilities that must be considered and simplifies the task of concurrent programming (for more, see https://glyph.twistedmatrix.com/2014/02/unyielding.html).

In Python, the closest equivalent to `go foo()` is `threading.Thread(target=foo).start()`. Or you can use gevent to get goroutine-like lightweight threads, but I find this is works better in go than in python because the entire language is designed around it. In python it's common to find libraries that are not compatible with gevent's monkey-patching and so you lose concurrency in a way that is difficult to detect or guard against.

Going to have to disagree with the criticism of gevent. The monkey-patching is very robust, despite the fact that monkey-patching is a sin in general. I've never had any problems with it, with the exception of third party native modules.

If code is using the regular Python socket and threading libraries, it will almost always work seamlessly. Gevent provides the closest thing to goroutines.

Third party native modules are exactly the issue. Many database drivers (etc) are wrappers around native libraries that do not integrate with gevent; the ease of integration with native code is generally one of Python's great strengths. To use gevent you must pay a lot more attention to implementation details of libraries you rely on.

> Are there any downsides to having to explicitly mark an async function using the "async def" syntax?

Yes. One main one is splintering the library ecosystem. So some libraries are using Twisted, some are using base threading system, some run in Tornado, some will use async and so on. They are usually not mixable. One day in the distant future they will all support async, but that means updating all of them.

Special markers ("async" waits points/objects) for IO co-routines tend to propagate vertically through all the API layers. If, say, a low level library you found is returning a Deferred/Promise/Future or generates values, then top level has to handle that as well. Then its parent also has to handle it.

It is infecting the ecosystem with low levels details about how the IO needs special casing because of how the GIL works. Yes, it is more elegant in Go, Rust, Erlang, C#, Java etc.

BTW Python has something like it based on the greenlet library (eventlet and gevent libraries are based on). Those monkeypatch system libraries so it manages to hide this complexity, but there are other costs, unsupported or untested side-effects being one.

What does the GIL have to do with this? All blocking I/O calls release the GIL.

It's more complicated than that. Specifically having a mix of any CPU and I/O bound threads doesn't play out as nicely:

See this:

http://dabeaz.blogspot.com/2010/02/revisiting-thread-priorit...

Or even better for a demo watch David Beazley's Pycon 2015 video (It is an awesome video even if you don't care about the GIL or socket programming).

Looks like this is the video referred to: http://pyvideo.org/video/3432/python-concurrency-from-the-gr...

Yap, thank you for finding. I was just being lazy.

Think of async/await as syntactic sugar for doing the kind of asynchronous programming when you pass callbacks/continuations and have an explicit event loop. For example, assume that you have an http_request function that looks like this:

    http_request(url, callback)

If you want to make a request, get a value, and then make another request depending on that value, you would need something like:

    def callback(result):
        url2 = get_from_value(result)

        def inner_callback(result):
            print('Final result:', result)

        http_request(url2, print)

    http_request(url1, callback)

This is very complicated, specially because in Python it's not easy to inline callbacks like in JavaScript. And even then, it's sometimes messy and becomes a callback hell.

With async/await you could do something like:

    result1 = await http_request(url1)
    result2 = await http_request(get_from_value(result1))
    print('Final result:', result2)

The await keyword doesn't block like in Perl 6. It suspends the execution and returns the control to the even loop. Then the event loop calls other functions in response to other events. All this happens in a single process and a single thread.

This model is more explicit than the Goroutines of Go or the multiple threads of Perl 6. But it has the advantage that the user has more control over when the execution control flows between different parts of the program. For example, if I have this code:

    await request1()
    # some synchronous code
    await request2()

I am completely sure that nothing will run while the code in the middle of the two request is running. This simplifies a lot of synchronization problems.

AFAIU await can only be used with async functions:

  async def http_request(url):
    # ...

  await http_request(u)

If the method I want to use is not an async one, is it possible to dynamically define a lambda ?