PyCon X


2-5 maggio 2019

Applying serverless architecture pattern to distributed data processing

Serverless architectures refer to applications that significantly depend on “cloud” services (knows as Backend as a Service) or on custom code that’s run in ephemeral runtime (Function as a Service or “FaaS").

To application developers, “serverless” mean app where some certain logic of it is still written by the developer but unlike traditional architectures or microservices is run in stateless compute runtime that is event-triggered, may only last for one invocation, and fully managed by a cloud. Serverless helps developers to transfer responsibility for keeping their apps up and running as well as scaling out their workload capacity without involving DevOps/ops as we got used to.

In this talk we will go through whole “serverless” thing: from decomposing app and its logic to microservices and further to smaller bits, i.e. functions to defining data flow through functions and building their fault-tolerant pipeline. Moreover, we will go through a demo that highlights key takeaways:

  • what are functions, what it could and could not be

  • how to design scalable architecture without getting into troubles by hitting concrete bottlenecks

  • how serverless can help scaling in/out compute capacity for data processing

The demo itself will include examples of applying serverless architecture pattern to emotion recognition app based on TensorFlow and OpenCV 3.3


  1. Gravatar
    Hi Denis,

    can you check the bulletlist formatting? They are all on the same line... try adding a blank line after each point.

    — Roberto Polli,
  2. Gravatar
    Hi Denis,
    reading your proposal, it seems to be more compliant with a devops track instead of pydata.
    What do u think about? Can I move it?

    — Ernesto Arbitrio,
  3. Gravatar
    Ernesto, well, this talk sits in-between this tracks: operations, development, data processing.

    I'm looking forward finding proper place for this talk (here's first iteration of the talk i made in PyCon Hong Kong
    — Denis Makogon,
  4. Gravatar
    FYI, in the definitive schedule we will have a track called "PyWeb&DevOps" where place all the talks related to DevOps and Web subjects. So if you consider that the topics are in line with the pydata track i'll not change it, otherwise let me know and I'll move your contribution in the PyWeb&DevOps track.
    — Ernesto Arbitrio,
  5. Gravatar
    Okay, i think that would be the right place because it really touches all of those aspects: data, web, operations. So, yeah, as per your thought, we better move this talk to the corresponding section.
    — Denis Makogon,
  6. Gravatar
    Ok! I moved it!
    Break a leg! :D
    — Ernesto Arbitrio,

Nuovo commento