Show filters Hide filters

Refine your search

Publication Year
1-36 out of 120 results
Change view
  • Sort by:
49:44 EuroPython English 2014

Python's Role in Big Data Analytics: Past, Present, and Future

Travis Oliphant - Python's Role in Big Data Analytics: Past, Present, and Future Python has had a long history in Scientific Computing which means it has had the fundamental building blocks necessary for doing Data Analysis for many years. As a result, Python has long played a role in scientific problems with the largest data sets. Lately, it has also grown in traction as a tool for doing rapid Data Analysis. As a result, Python is the center of an emerging trend that is unifying traditional High Performance Computing with "Big Data" applications. In this talk I will discuss the features of Python and its popular libraries that have promoted its use in data analytics. I will also discuss the features that are still missing to enable Python to remain competitive and useful for data scientists and other domain experts. Finally, will describe open source projects that are currently occupying my attention which can assist in keeping Python relevant and even essential in Data Analytics for many years to come.
  • Published: 2014
  • Publisher: EuroPython
  • Language: English
1:22:46 EuroPython English 2014

EuroPython 2014: July 22, 2014 - Lightning Talks

Lightning Talks: Elliptics: Anton Tyurin, On Being Super: Austin Bingham, PYCON Finland: Mikko Ohtamaa, Regularities in language: Dmitrijs Milajevs, pip install pyladies: Lynn Root, Iktomi Forms: Harut Dagesyan, Killer Robots: Radomir Dopieralski, ZeroVM: Lars Butler, Zombies and Application Runtimes: Dmitry Jemerov, FritzConnection - Communicate with the AVM Fritz Box: Klaus Bremer, Argment Clinic: Larry Hastings, PEP 473: Adding structured data to builtin exceptions: Sebastian Kreft, Supporting Python 2 and Python 3 witth the same source code: Stefan Schwarzer, Birthday: Mark Shannon, nsupdate.info bepasty: Thomas Waldmann, Python Core Security: Christian Heimes, Hands On Unix: Rok Garbas, Deproulette: Joar Wandborg
  • Published: 2014
  • Publisher: EuroPython
  • Language: English
1:16:08 EuroPython English 2014

EuroPython 2014: July 25, 2014 - Lightning Talks

Lightning Talks: Sparts - Quick service Prototyping: Manu, Blinky's Async Adventure: Petr Viktorin, Tryton Unconference Leipzig 2014: Udono, Yet another conference announcement: Baptiste Mispelon, Configurable omnipotent custom applications integrated network engine: Anton Tyurin, What we did at #ep14: "vtemian" Vlad Temian, Foreman client: Lukas Bednar, HSTS: Abraham Martin, PyAr Pablo G. Celayes, Celia Cintas, Python Quantum Platform: Yves J. Hilpisch, CodeWeek.eu: Erika Pogorelc, Our beautiful minds: Remco Wendt, Stackless - Best Ever: Christian Tismer, Stella: Compiling a small subset of Python: David Mohr, Plone Mosiac Sprint Barcelona: Philip Bauer
  • Published: 2014
  • Publisher: EuroPython
  • Language: English
48:00 EuroPython English 2014

The Cython Compiler for Python

Stefan Behnel - The Cython Compiler for Python The Cython compiler is the most widely used static compiler for Python. It is used to speed up Python code and to extend CPython with fast native extension modules that process huge amounts of data all around the world. This talk by one of the core developers gives an intro to using the compiler and an overview of its major features. ----- The Cython compiler is the most widely used static compiler for Python. The code it generates is used in countless critical applications that process huge amounts of data world wide. Cython has two major use cases: to compile Python code into fast native extension modules, and to connect native code to the CPython runtime. The main goal of the Cython project is to make it easy for users to manually optimise their Python code to make it run at C speed. This talk by one of the core developers will give an intro to using the compiler and an overview of its major features. Outline will be more or less as follows: * Cython: intro to the project and the compiler (4 min.) * compiling Python code - how to do it and what you get (3 min.) - a tiny bit of distutils (2 min.) * static typing and Cython extensions to the Python language - static typing in Cython language syntax (3 min.) - static typing in pure Python syntax (2 min.) - why Cython's type system is cool and what users need to know about it (8 min.) - Cython for optimising Python code (5 min.) * quick intro: talking to native C/C++ code in Cython - using external C APIs (4 min.) - using external C++ APIs (3 min.) - how to build and link in distutils (2 min.) - notes on ways to wrap large C-APIs (1 min.) * quick overview: special features for high-performance code - NumPy integration and memory views, fused types, parallel loops in all brevity (3 min.)
  • Published: 2014
  • Publisher: EuroPython
  • Language: English
48:27 EuroPython English 2014

Performance Python for Numerical Algorithms

Yves - Performance Python for Numerical Algorithms This talk is about several approaches to implement high performing numerical algorithms and applications in Python. It introduces into approaches like vectorization, multi-threading, parallelization (CPU/GPU), dynamic compiling, high throughput IO operations. The approach is a practical one in that every approach is illustrated by specific Python examples. The talk uses, among others, the following libraries: * NumPy * numexpr * IPython.Parallel * Numba * NumbaPro * PyTables
  • Published: 2014
  • Publisher: EuroPython
  • Language: English
27:50 EuroPython English 2014

Twisted Names

Richard Wall - Twisted Names: DNS Building Blocks for Python Programmers In this talk I will report on my efforts to update the DNS components of Twisted and discuss some of the things I've learned along the way. I'll demonstrate the EDNS0, DNSSEC and DANE client support which I have been working on and show how these new Twisted Names components can be glued together to build novel DNS servers and clients. Twisted is an event-driven networking engine written in Python and licensed under the open source MIT license. It is a platform for developing internet applications.
  • Published: 2014
  • Publisher: EuroPython
  • Language: English
23:34 EuroPython English 2014

Jigna: a seamless Python-JS bridge to create rich HTML UIs for Python apps

Prashant Agrawal - Jigna: a seamless Python-JS bridge to create rich HTML UIs for Python apps Jigna aims to provide an easy way to create rich user interfaces for Python applications using web technologies like HTML, CSS and Javascript, as opposed to widget based toolkits like Qt/wx or native toolkits. It provides a seamless two-way data binding between the Python model and the HTML view by creating a Python-JS communication bridge. This ensures that the view is always live as it can automatically update itself when the model changes, and update the model when user actions take place on the UI. The Jigna view can be rendered in an in-process Qt widget or over the web in a browser.
  • Published: 2014
  • Publisher: EuroPython
  • Language: English
39:57 EuroPython English 2014

Support Python 2 and 3 with the same code

Stefan Schwarzer - Support Python 2 and 3 with the same code Your library supports only Python 2, - but your users keep nagging you about Python 3 support? As Python 3 gets adopted more and more, users ask for Python 3 support in existing libraries for Python 2. This talk mentions some approaches for giving users a Python 3 version, but will quickly focus on using the very same code for a Python 2 and a Python 3 version. This is much easier if you require Python 2.6 and up, and yet a bit easier if you require Python 3.3 as the minimum Python 3 version. The talk discusses main problems when supporting Python 3 (some are easily solved): * `print` is a function. * More Python APIs return iterators that used to return lists. * There's now a clear distinction between bytes and unicode (text) strings. * Files are opened as text by default, requiring an encoding to apply on reading and writing. The talk also explains some best practices: * Start with a good automatic test coverage. * Deal with many automatic conversions with a one-time 2to3 run. * Think about how your library should handle bytes and unicode strings. (Rule of thumb: Decode bytes as early as possible; encode unicode text as late as possible.) * Should you break compatibility with your existing Python 2 API? (Yes, if there's no other way to design a sane API for Python 2 and 3. If you do it, raise the first part of the version number.) * Try to keep code that's different for Python 2 and 3 minimal. Put code that needs to be different for Python 2 and 3 into a `compat` module. Or use third-party libraries like `six` or `future`. Finally, the talk will mention some helpful resources on the web.
  • Published: 2014
  • Publisher: EuroPython
  • Language: English
40:38 EuroPython English 2014

Design considerations while Evaluating, Developing, Deploying a distributed task processing system

konarkmodi - Design considerations while Evaluating, Developing, Deploying a distributed task processing system With the growing world of web, there are numerous use-cases which require tasks to be executed in an asynchronous manner and in a distributed fashion. Celery is one of the most robust, scalable, extendable and easy-to-implement frameworks available for distributed task processing. While developing applications using Celery, I have had considerable experience in terms of what design choices one should be aware of while evaluating an existing system or developing one's own system from scratch.
  • Published: 2014
  • Publisher: EuroPython
  • Language: English
22:39 EuroPython English 2014

3D sensors and Python: A space odyssey

Celia - 3D sensors and Python: A space odyssey This talk will show how to build a simple open source based NUI (Natural User Interface) game with 3D Sensors, incorporating PyOpenNI with PyGame and WebGL. OpenNI allows you operate several 3D sensors, enabling hardware independent game development (supported 3D sensors are Microsoft Kinect, PrimeSense Carmine or Asus XTion). It also runs on Linux, Mac OS X and Windows. ----- This talk will start with a brief introduction to 3D Sensors and OpenNI. Then we’ll surf into PyOpenNI, features such as the skeleton, hand and gesture tracking, RGB and depth video. Every topic will be presented with practical demos. The talk will end with a demo integrating WebGL (THREE.JS), 3D sensors, Flask and ZMQ to produce a simple fully open source based NUI game. Attendees will not only learn about game related technologies but also about innovative ways of doing domotics, cinema & art, Interactive visualization, scientific research, educations, etc. 3D Sensors will be available for testing during the event - you can get yours for about 80 to 140 Euros (depending on the brand). Slides and demo code will be available at Github. Talk structure: * Introduction: hardware and OpenNI goodies and a tale of PCL (5’) * Hands On PyOpenNI * Normal and Depth camera - basics concepts and small demo (5’) * Skeleton - basics concepts and small demo. (5’) * Hand & gesture - basics concepts and small demo. (5’) * Final Demo * What we’re going to use? Flask, ZMQ, THREE.JS, PyOpenNI. (6’) * Q&A. (4’)
  • Published: 2014
  • Publisher: EuroPython
  • Language: English
39:59 EuroPython English 2014

Systems Integration: The OpenStack success story

Flavio Percoco - Systems Integration: The OpenStack success story OpenStack is a huge, open-source cloud provider. One of the main tenets of OpenStack is the (Shared Nothing Architecture) to which all modules stick very closely. In order to do that, services within OpenStack have adopted different strategies to integrate themselves and share data without sacrificing performance nor moving away from SNA. This strategies are not applicable just to OpenStack but to any distributed system. Sharing data, regardless what that data is, is a must-have requirement of any successful cloud service. This talk will present some of the existing integration strategies that are applicable to cloud infrastructures and enterprise services. The talk will be based on the strategies that have helped OpenStack to be successful and most importantly, scalable. Details ====== Along the lines of what I've described in the abstract, the presentation will walk the audience through the state of the art of existing system integration solutions, the ones that have been adopted by OpenStack and the benefits of those solutions. At the end of the talk, a set of solutions under development, ideas and improvements to the existing ones will be presented. The presentation is oriented to distributed services, fault-tolerance and replica determinism. It's based on a software completely written in python and running successfully on several production environments. The presentation will be split in 3 main topics: Distributed System integration ----------------------------------- * What's it ? * Why is it essential for cloud infrastructures? * Existing methods and strategies OpenStack success story ---------------------------- * Which methods did OpenStack adopt? * How / Why do they work? * What else could be done? Coming Next --------------- * Some issues of existing solutions * What are we doing to improve that? * Other solutions coming up
  • Published: 2014
  • Publisher: EuroPython
  • Language: English
40:20 EuroPython English 2014

Rethinking packaging, development and deployment

Domen Kožar - Rethinking packaging, development and deployment In Python, we're trying to solve packaging problems in our own domain, but maybe someone else already solved most our problems. In the talk I'll show how I develop and deploy Python projects that can be easily mixed with non-Python dependencies. "Nix" will be demonstrated to replace technologies in our stack: pip, virtualenv, buildout, ansible, jenkins. ----- Python is often mixed with other languages in development stack, nowadays it's hard to escape any JavaScript dependencies. If you add some C dependencies such as GStreamer to the stack, packaging becomes a burden. While tweaking our packaging infrastructure will make things better, it's hard to fix fundamental problem of packaging with current ad-hoc solutions in Python domain. Using Nix for about a year gave me an insight that solving packaging problem at operating system level (bottom-up) is a better approach. For example, wouldn't it be cool to have "virtualenv" implemented inside your package manager, so you could isolate also non-Python dependencies and not just Python packages for your project and not worry if system was updated? We'll also show what benefits do we get by using the same tool for development and deployment and how little we have to do to deploy our application.
  • Published: 2014
  • Publisher: EuroPython
  • Language: English
36:34 EuroPython English 2014

Designing NRT(NearRealTime) stream processing systems: Using python with Storm and Kafka

konarkmodi - Designing NRT(NearRealTime) stream processing systems: Using python with Storm and Kafka The essence of near-real-time stream processing is to compute huge volumes of data as it is received. This talk will focus on creating a pipeline for collecting huge volumes of data using Kafka and processing for near-real time computations using Storm.
  • Published: 2014
  • Publisher: EuroPython
  • Language: English
31:10 EuroPython English 2014

Python Debugger Uncovered

Dmitry Trofimov - Python Debugger Uncovered This talk will explain how to implement a debugger for Python. We'll start with setting a simple trace function, then look how it is implemented in modern IDEs like PyCharm and Pydev. Then we go further in the details and uncover the tricks used to implement some cool features like exception handling and multiprocess debugging. ----- Presentation describes how to implement debugger for Python and has 4 parts: * Tracing Python code Explains how to use trace function * Debugger Architecture Explains which parts consists of a modern full-fledged debugger. * A Bit of Details Explains how to make code to work for all python versions and implementations, survive gevent monkey-patching etc. * Cool Features Explains how to implement exception handling and multiprocess debugging
  • Published: 2014
  • Publisher: EuroPython
  • Language: English
20:47 EuroPython English 2014

Automatic code reviews

Carl Crowder - Automatic code reviews A lot of great Python tools exist to analyse and report on your codebase, but they can require a lot of initial set up to be useful. Done right, they can be like an automatic code review. This talk will explain how to set up and get the best out of these tools, especially for an existing, mature codebase. ----- Static analysis tools are a great idea in theory, but are not often really used in practice. These tools usually require quite a lot of initial effort to get set up in a way which produces meaningful output for you or your organisation's particular coding style and values. As a result, it's common to see initial enthusiasm replaced by ignoring the tools. Such tools can be incredibly beneficial however, and even go so far as to provide an automatic code review, and this talk will explain what kind of benefits you can get from the tools, as well as explain what you can and cannot expect. This talk is aimed at experienced developers who are interested in improving their coding practices but who have either never tried static analysis tools, or who have not seen the upsides. It will hopefully also be useful to people who do use the tools, perhaps introducing them to new tools or concepts they were not aware of yet.
  • Published: 2014
  • Publisher: EuroPython
  • Language: English
26:10 EuroPython English 2014

Python for Zombies: 15.000 enrolled in the first Brazilian MOOC to teach Python

Ashikaga - Python for Zombies: 15.000 enrolled in the first Brazilian MOOC to teach Python Experiences of how we spread the Python community in Brazil with a non english MOOC (Massive Open Online Course) to teach programming. Hacking basic modules and classes to obtain the "Answer to the Ultimate Question of Life, the Universe, and Everything". A funny way to teach programming.
  • Published: 2014
  • Publisher: EuroPython
  • Language: English
32:21 EuroPython English 2014

Extending Python, what is the best option for me?

Francisco Fernández Castaño - Extending Python, what is the best option for me? Python is a great language, but there are occasions where we need access to low level operations or connect with some database driver written in C. With the FFI(Foreign function interface) we can connect Python with other languages like C, C++ and even the new Rust. There are some alternatives to achieve this goal, Native Extensions, Ctypes and CFFI. I'll compare this three ways of extending Python. ----- In this talk we will explore all the alternatives in cpython ecosystem to load external libraries. In first place we'll study the principles and how shared libraries work. After that we will look into the internals of CPython to understand how extensions work and how modules are loaded. Then we will study the main three alternatives to extend CPython: Native Extensions, Ctypes and CFFI and how to automate the process. Furthermore we will take a look to other python implementations and how we can extend it.
  • Published: 2014
  • Publisher: EuroPython
  • Language: English
26:31 EuroPython English 2014

Advanced Uses of py.test Fixtures

Floris Bruynooghe - Advanced Uses of py.test Fixtures One unique and powerful feature of py.test is the dependency injection of test fixtures using function arguments. This talk aims to walk through py.test's fixture mechanism gradually introducing more complex uses and features. This should lead to an understanding of the power of the fixture system and how to build complex but easily-managed test suites using them. ----- This talks will assume some basic familiarity with the py.test testing framework and explore only the fixture mechanism. It will build up more complex examples which will lead up to touching on other plugin features of py.test. It is expected people will be familiar with python features like functions as first-class objects, closures etc.
  • Published: 2014
  • Publisher: EuroPython
  • Language: English
22:31 EuroPython English 2014

Out-of-Core Columnar Datasets

Francesc Alted - Out-of-Core Columnar Datasets Tables are a very handy data structure to store datasets to perform data analysis (filters, groupings, sortings, alignments...). But it turns out that how the tables are actually implemented makes a large impact on how they perform. Learn what you can expect from the current tabular offerings in the Python ecosystem. ----- It is a fact: we just entered in the Big Data era. More sensors, more computers, and being more evenly distributed throughout space and time than ever, are forcing data analyists to navigate through oceans of data before getting insights on what this data means. Tables are a very handy and spreadly used data structure to store datasets so as to perform data analysis (filters, groupings, sortings, alignments...). However, the actual table implementation, and especially, whether data in tables is stored row-wise or column-wise, whether the data is chunked or sequential, whether data is compressed or not, among other factors, can make a lot of difference depending on the analytic operations to be done. My talk will provide an overview of different libraries/systems in the Python ecosystem that are designed to cope with tabular data, and how the different implementations perform for different operations. The libraries or systems discussed are designed to operate either with on-disk data ([PyTables], [relational databases], [BLZ], [Blaze]...) as well as in-memory data containers ([NumPy], [DyND], [Pandas], [BLZ], [Blaze]...). A special emphasis will be put in the on-disk (also called out-of-core) databases, which are the most commonly used ones for handling extremely large tables. The hope is that, after this lecture, the audience will get a better insight and a more informed opinion on the different solutions for handling tabular data in the Python world, and most especially, which ones adapts better to their needs.
  • Published: 2014
  • Publisher: EuroPython
  • Language: English
25:36 EuroPython English 2014

Managing the Cloud with a Few Lines of Python

Frank - Managing the Cloud with a Few Lines of Python One of the advantages of cloud computing is that resources can be enabled or disabled dynamically. E. g. is an distributed application short on compute power one can easily add more. But who wants to do that by hand? Python is a perfect fit to control the cloud. The talk introduces the package Boto which offers an easy API to manage most of the Amazon Web Services (AWS) as well as a number of command line tools. First some usage examples are shown to introduce the concepts behind Boto. For that a few virtual hosts with different configurations are launched, and the use of the storage service S3 is briefly introduced. Based on that a scalable continuous integration system controlled by Boto is developed to show how easy all the required services can be used from Python. Most of the examples will be demonstrated during the talk. They should be easily adoptable for similar use cases or serve as an starting point for more different ones. ----- One of the advantages of cloud computing is that resources can be enabled or disabled dynamically. E. g. is an distributed application short on compute power one can easily add more. But who wants to do that by hand? Python is a perfect fit to control the cloud. The talk introduces the package Boto which offers an easy API to manage most of the Amazon Web Services (AWS) as well as a number of command line tools. First some usage examples are shown to introduce the concepts behind Boto. For that a few virtual instances with different configurations are launched, and the use of the storage service S3 is briefly introduced. Based on that a scalable continuous integration system controlled by Boto is developed to show how easy all the required services can be used from Python. Most of the examples will be demonstrated during the talk. They should be easily adoptable for similar use cases or serve as an starting point for more different ones.
  • Published: 2014
  • Publisher: EuroPython
  • Language: English
25:31 EuroPython English 2014

Graph Databases, a little connected tour

Francisco Fernández Castaño - Graph Databases, a little connected tour There are many kinds of NoSQL databases like, document databases, key-value, column databases and graph databases. In some scenarios is more convenient to store our data as a graph, because we want to extract and study information relative to these connections. In this scenario, graph databases are the ideal, they are designed and implemented to deal with connected information in a efficient way. ----- There are many kinds of NoSQL databases like, document databases, key-value, column databases and graph databases. In some scenarios is more convenient to store our data as a graph, because we want to extract and study information relative to these connections. In this scenario, graph databases are the ideal, they are designed and implemented to deal with connected information in a efficient way. In this talk I'll explain why NoSQL is necessary in some contexts as an alternative to traditional relational databases. How graph databases allow developers model their domains in a natural way without translating these domain models to an relational model with some artificial data like foreign keys and why is more efficient a graph database than a relational one or even a document database in a high connected environment. Then I'll explain specific characteristics of Neo4J as well as how to use Cypher the neo4j query language through python.
  • Published: 2014
  • Publisher: EuroPython
  • Language: English
25:50 EuroPython English 2014

Jython in practice

Fredrik Håård - Jython in practice A lot of people have heard of Jython, some have tried it, but it seems few have actually deployed it in a corporate environment. In this talk I'll share my experiences in using Jython as a testbed for Java applications, for rapid prototyping in Java desktop and web environments, and for embedding scripting capabilities in Java products. ----- Not everyone gets paid to work with Python all the time, but if you find yourself in a Java project, there are good chances you could benefit from Python without throwing out the Java stack. Using Jython, you can do rapid prototyping without the long edit-compile-test cycles normally associated with large Java projects, whether on the web or the desktop, and when testing an application might become a nightmare of scaffolding in Java, a little Jython may be just what you need to be able to run your tests smoothly. At the end of this talk, I will put on my politician´s hat and bring up the best - and worst - arguments to use to get permission to use Jython in a corporate environment.
  • Published: 2014
  • Publisher: EuroPython
  • Language: English
39:01 EuroPython English 2014

Metaprogramming, from decorators to macros

Andrea Crotti - Metaprogramming, from decorators to macros Starting off with the meaning of metaprogramming we quickly dive into the different ways Python allows this. First we talk about class and functions decorators, when decorators are not enough anymore we'll explore the wonders of metaclasses. In the last part of the talk we'll talk about macros, first in Lisp and then using the amazing macropy library. ----- This talk is a journey in the wonderful world of metaprogramming. We start off with the meaning of metaprogramming and what it can be used for. Then we look at what can be done in Python, introducing function and class decorators. When decorators are not enough anymore we move to the black magic of metaclasses, showing how we can implemement a simple Django-like model with them. In the bonus track we'll talk about macros, as the ultimate metaprogramming weapon, showing briefly how Lisp macros work and introducing the amazing [macropy library].
  • Published: 2014
  • Publisher: EuroPython
  • Language: English
26:53 EuroPython English 2014

Post-Mortem Debugging with Heap-Dumps

Anselm Kruis - Post-Mortem Debugging with Heap-Dumps UNIX core-dumps, Windows minidumps or Java heap-dumps are well established technologies for post-mortem defect analysis. I'll present a similar technology for Python. An improved pickling mechanism makes it possible to serialise the state of a Python program for subsequent analysis with a conventional Python-debugger. ----- Post-Mortem Debugging with Heap-Dumps === UNIX core-dumps, Windows minidumps and analogous solutions of other operating systems are well established technologies for post-mortem defect analysis of native-code processes. In principle those dumps can be used to analyse „interpreted“ programs running within a native-code interpreter-process. However in practise this approach is tedious and not always successful. Therefore operating system independent dump methods were developed for some „interpreted“ languages. A prominent example are Java heap dumps. Unfortunately up to now there was no practically usable dump-method for Python. Various attempts were made to utilise OS-level dump methods. In 2012 Eli Finer published the Python module *pydump*. This module pickles the traceback of an exception and subsequently uses the pdb debugger to analyse the unpickled traceback. Unfortunately *pydump* fails on PicklingErrors. In my talk I'll present the Python package [*pyheapdump*]. It has the same operation principle as Eli's *pydump*, but is an independent implementation. *pyheapdump* uses an extended pickler ([sPickle] to serialise all relevant objects of a Python process to a file. Later on a fault tolerant unpickler recreates the objects and a common Python debugger can be used to analyse the dump. The pickler extensions make it possible to: * pickle and unpickle many commonly not pickleable objects. * replace the remaining not pickleable objects by surrogate objects so that the resulting object graph is almost isomorphic to the original object graph. Which objects are relevant? In its default operation mode *pyheapdump* uses the frame-stacks of all threads as start point for pickling. Following the usual rules for pickling the dump includes all local variables and all objects reachable from a local variable and so on. That is usually enough for a successful defect analysis. Compared with other Python post-mortem debugging methods *pyheapdump* has several advantages: * It is a pure Python solution and independent from the operation system. * Creation of the pyheapdump and fault analysis can be performed different computers. * It is not obstructive. It does not modify / monkey-patch or disturb the dumped process in any way, with the exception of loading additional modules. * If used with the Pydev-debugger, it supports multi-threaded applications. * If used with the Pydev-debugger and Stackless Python, it supports tasklets. The implementation of *pyheapdump* is fairly small, because it draws most of its functionality from the underlying sPickle package and from the new Stackless-Support of the Pydev-Debugger. Therefore it is - despite of its short history - already a useful piece of software. Outline of the talk --- 1. Introduction to the problem 2. Previous works 3. The concept of *pyheapdump* 4. Live demonstration 5. Open problems and further development 6. Questions and Answers
  • Published: 2014
  • Publisher: EuroPython
  • Language: English
46:46 EuroPython English 2014

Will I still be able to get a job in 2024 if I don't do TDD?

Emily Bache - Will I still be able to get a job in 2024 if I don't do TDD? Geoffrey Moores's book "Crossing the chasm" outlines the difficulties faced by a new, disruptive technology, when adoption moves from innovators and visionaries into the mainstream. Test Driven Development is clearly a disruptive technology, that changes the way you approach software design and testing. It hasn't yet been embraced by everyone, but is it just a matter of time? Ten years from now, will a non-TDD practicing developer experience the horror of being labelled a technology adoption 'laggard', and be left working exclusively on dreadfully boring legacy systems? It could be a smart move to get down to your nearest Coding Dojo and practice TDD on some Code Katas. On the other hand, the thing with disruptive technologies is that even they can become disrupted when something better comes along. What about Property-Based Testing? Approval Testing? Outside-In Development? In this talk, I'd like to look at the chasm-crossing potential of TDD and some related technologies. My aim is that both you and I will still be able to get a good job in 2024. ----- TDD hasn't yet been embraced by everyone, but is it just a matter of time? Ten years from now, will a non-TDD practicing developer experience the horror of being labelled a technology adoption 'laggard', and be left working exclusively on dreadfully boring legacy systems? It could be a smart move to get down to your nearest Coding Dojo and practice TDD on some Code Katas. On the other hand, the thing with disruptive technologies is that even they can become disrupted when something better comes along. What about Property-Based Testing? Approval Testing? Outside-In Development? In this talk, I'd like to look at the chasm-crossing potential of TDD and some related technologies. My aim is that both you and I will still be able to get a good job in 2024.
  • Published: 2014
  • Publisher: EuroPython
  • Language: English
44:53 EuroPython English 2014

How Pony ORM translates Python generators to SQL queries

Alexey Malashkevich - How Pony ORM translates Python generators to SQL queries Pony ORM is an Object-Relational Mapper implemented in Python. It uses an unusual approach for writing database queries using Python generators. Pony analyzes the abstract syntax tree of a generator and translates it to its SQL equivalent. The translation process consists of several non-trivial stages. In this talk one of Pony ORM authors will reveal the internal details of this process. ----- [Pony ORM] is an object-relational mapper implemented in Python. It allows writing advanced queries to a database using plain Python in the form of a generator expression. This way queries look very concise. The main feature of Pony is to provide a method to write declarative queries to databases in pure Python using generators. For this purpose Pony analyzes the abstract syntax tree of a generator and translates it to its SQL equivalent. Following is a sample of a query in Pony: select(p for p in Product if "iPad" in p.name and p.price >= 500) This query translates to SQL using a specific database dialect. Currently Pony works with SQLite, MySQL, PostgreSQL and Oracle databases. In this talk one of Pony ORM authors will go through the process of the query translation and dig into the implementation details. Attendees are going to walk away with the understanding of: 1. Principles of building a programming language translator 2. Python to SQL translator implementation details 3. Approaches for creating a pluggable translator architecture The presentation outline: - Why Python generators are good for representing SQL queries - Main stages of Python to SQL translation overview - Decompiling Python bytecode into Python AST - Translating Python AST to database-independent SQL representation - Generating SQL for specific database - Pluggable translator architecture - Performance concerns: is such way of building SQL slower or faster then Django's and SQLAlchemy's?
  • Published: 2014
  • Publisher: EuroPython
  • Language: English
40:31 EuroPython English 2014

Pythonista: A full-featured Python environment for iOS devices

Chris Clauss - Pythonista: A full-featured Python environment for iOS devices The Pythonista app delivers a full-featured Python development experience on an iPad or an iPhone. This introduction to the app will provide a rapid overview of the Pythonista user experience, features and Community Forum. Then it will focus on a few source code examples of using the GPS to deliver real-time local weather, use the image library to manipulate images and convert documents, use the gyroscope to understand pitch, yaw, and roll, use Dropbox to backup and restore scripts, images, etc.
  • Published: 2014
  • Publisher: EuroPython
  • Language: English
44:39 EuroPython English 2014

Stackless: Recent advancements and future goals

Christian Tismer/Anselm Kruis - Stackless: Recent advancements and future goals Stackless (formerly known as Stackless-Python) is an enhanced variant of the Python-language. Stackless is best known for its lightweight microthreads. But that's not all. In this talk Stackless core developers demonstrate recent advancements regarding multi-threading, custom-scheduling, debugging with Stackless and explain future plans for Stackless. ----- Stackless: Recent advancements and future goals ------------------------------------------------------- Since Python release 1.5 Stackless Python is an enhanced variant of C-Python. Stackless is best known for its addition of lightweight microthreads (tasklets) and channels. Less known are the recent enhancements that became available with Stackless 2.7.6. In this talk core Stackless developers demonstrate * The improved multi-threading support * How to build custom scheduling primitives based on atomic tasklet operations * The much improved debugger support * ... Stackless recently switched the new master repository from hg.python.org/stackless to bitbucket to allow for a more open development process. We'll summarise our experience and discuss our plans for the future development of Stackless. The talk will be help by Anselm Kruis and Christian Tismer.
  • Published: 2014
  • Publisher: EuroPython
  • Language: English
40:36 EuroPython English 2014

Scaling with Ansible

Federico Marani - Scaling with Ansible Ansible is a powerful DevOps swiss-army knife tool, very easy to configure and with many extensions built-in. This talk will quickly introduce the basics of Ansible, then some real-life experience tips on how to use this tool, from setting up dev VMs to multi-server setups. ----- Infrastructure/Scaling is a topic really close to me, I'd like to have the chance to talk about how we set this up in the company I work for. Our infrastructure is around 10-15 servers, provisioned on different cloud providers, so a good size infrastructure. Presentation is going to be divided in 3 parts, first part is going to be focused on comparing sysadmin and devops, then there will be an introduction to the basic concepts of Ansible. I want to spend most on the last part, which is going to give some tips based on our experience with it. Many ideas will come from this presentation which i gave at DJUGL in London, with a longer session I will have more chances to delve into more detail, especially on how we use it, from vagrant boxes setup to AWS and DigitalOcean boxes, network configuration, software configurations, etc... I want to offer as many real-life tips as possible, without going too much offtopic as far as Ansible is concerned
  • Published: 2014
  • Publisher: EuroPython
  • Language: English
28:48 EuroPython English 2014

The inner guts of Bitbucket

Erik van Zijst - The inner guts of Bitbucket Today Bitbucket is more than 30 times bigger than at the time of acquisition almost 4 years ago and serves repositories to over a million developers. This talk lays out its current architecture in great detail, from Gunicorn and Django to Celery and HA-Proxy to NFS. ----- This talk is about Bitbucket's architecture. Leaving no stone unturned, I'll be covering the entire infrastructure. Every component, from web servers to message brokers and load balancing to managing hundreds of terabytes of data. Since its inception in 2008, Bitbucket has grown from a standard, modest Django app into a large, complex stack that while still based around Django, has expanded into many more components. Today Bitbucket is more than 30 times bigger than at the time of acquisition almost 4 years ago and serves Git and Mercurial repos to over a million users and growing faster now than ever before. Our current architecture and infrastructure was shaped by rapid growth and has resulted in a large, mostly horizontally scalable system. What has not changed is that it's still nearly all Python based and could serve as inspiration or validation for other community members responsible for rapidly scaling their apps. This talk will layout the entire architecture and motivate our technology choices. From our Gunicorn to Celery and HA-Proxy to NFS.
  • Published: 2014
  • Publisher: EuroPython
  • Language: English
26:56 EuroPython English 2014

Using asyncio (aka Tulip) for home automation

Dougal Matthews - Using asyncio (aka Tulip) for home automation This talk will cover the new asyncio library in Python 3.4 (also known as Tulip) and will use the area of home automation as a case study to explore its features. This talk will be based on code using Python 3.3+. Home automation is a growing area and the number of devices and potential applications is huge. From monitoring electricity usage to the temperature inside or outside your house to remote control of lights and other appliances the options are almost endless. However, managing and monitoring these devices is typically a problem that works best with event driven applications. This is where asnycio comes in, it was originally proposed in PEP 3156 by our BDFL, Guido van Rossum. Asyncio aims to bring a clear approach to the python ecosystem and borrows from a number of existing solutions to come up with something clean and modern for the Python stdlib. ----- This talk will cover the new asyncio library in Python 3.4 (also known as Tulip) and will use the area of home automation as a case study to explore its features. This talk will be based on code using Python 3.3+. Home automation is a growing area and the number of devices and potential applications is huge. From monitoring electricity usage to the temperature inside or outside your house to remote control of lights and other appliances the options are almost endless. However, managing and monitoring these devices is typically a problem that works best with event driven applications. This is where asnycio comes in, it was originally proposed in PEP 3156 by our BDFL, Guido van Rossum. Asyncio aims to bring a clear approach to the python ecosystem and borrows from a number of existing solutions to come up with something clean and modern for the Python stdlib. This talk will introduce asyncio and use it within the context of home automation and dealing with multiple event driven devices. Therefore we will cover asyncio and the lessions learned from using different devices in this context. Some of the devices that will be used include: - Raspberry Pi - RFXCom's RFXtrx, USB serial tranciever. - Owl CM160 electricity tracker. - Oregon scientific thermometers. - Foscam IP cameras. This talk will also briefly cover the previous solution I used which was developed with Twisted and compare it briefly with my new code using asyncio.
  • Published: 2014
  • Publisher: EuroPython
  • Language: English
44:55 EuroPython English 2014

The Sorry State of SSL

Hynek - The Sorry State of SSL Those web pages with shiny lock icons boasting that your data is safe because of “256 bit encryption”? They are lying. In times of mass surveillance and commercialized Internet crime you should know why that’s the case. This talk will give you an overview that will help you to assess your personal security more realistically and to make your applications as secure as possible against all odds. ----- The rule of thumb for people without degrees in cryptography on securing data on the Internet is “GPG for data at rest. TLS for data in motion”. And it’s actually a very good rule everyone should follow. The only kicker though is that configuring (and using!) TLS properly is not as simple as it sounds and if you’re not diligent as a user, developer, and ops engineer, you can easily compromise your data’s security despite best effort of everyone else. This talk will be multifaceted; you will learn: - how SSL and TLS roughly work and why their state is sorry, - server- and client-side duties for best possible security, - what alternatives you have for using TLS in Python, - things to keep in mind when configuring servers, - and what perils outside your control still can trip you up. In other words, the leitmotif is to show you the most common traps you should know about when using and deploying applications relying on TLS for transport layer security and how to avoid them.
  • Published: 2014
  • Publisher: EuroPython
  • Language: English
24:43 EuroPython English 2014

How to Setup a new Python Project

Felix Wick/Florian Wilhelm - How to Setup a new Python Project Setting up a new Python project from scratch can be quite hard. How to structure your files and directories. Where should my packages, modules, documentation and unit tests go? How do I configure setup.py, Sphinx and so on? We provide proven answers! ----- Whenever a Python beginner starts with its own project he or she is confronted with the same technical questions. Questions about a well thought out directory structure to hold all the files. How setup.py needs to be configured and even what it is capable of like specifying entry points and other goodies. We show from the experience of our yearslong work with Python how to structure your Python project in terms of folders, files, modules and packages. How to configure setup.py to specify your requirements, to use it with nosetests, with Sphinx and so on. We also elaborate on the usage of Git and Versioneer to help you version your package.
  • Published: 2014
  • Publisher: EuroPython
  • Language: English
26:45 EuroPython English 2014

Packaging and testing with devpi and tox

holger krekel - packaging and testing with devpi and tox This talk discusses good ways to organise packaging and testing for Python projects. It walks through a per-company and an open source scenario and explains how to best use the "devpi-server" and "tox" for making sure you are delivering good and well tested and documented packages. As time permits, we also discuss in-development features such as real-time mirroring and search. ----- The talk discusses the following tools: - devpi-server for running an in-house or per-laptop python package server - inheritance between package indexes and from pypi.python.org public packages - the "devpi" client tool for uploading docs and running tests - running of tests through tox - summary view with two work flows: open source releases and in-house per-company developments - roadmap and in-development features of devpi and tox
  • Published: 2014
  • Publisher: EuroPython
  • Language: English
28:30 EuroPython English 2014

Scalable Realtime Architectures in Python

Jim Baker - Scalable Realtime Architectures in Python This talk will focus on you can readily implement highly scalable and fault tolerant realtime architectures, such as dashboards, using Python and tools like Storm, Kafka, and ZooKeeper. We will focus on two related aspects: composing reliable systems using at-least-once and idempotence semantics and how to partition for locality. ----- Increasingly we are interested in implementing highly scalable and fault tolerant realtime architectures such as the following: * Realtime aggregation. This is the realtime analogue of working with batched map-reduce in systems like Hadoop. * Realtime dashboards. Continuously updated views on all your customers, systems, and the like, without breaking a sweat. * Realtime decision making. Given a set of input streams, policy on what you like to do, and models learned by machine learning, optimize a business process. One example includes autoscaling a set of servers. Obvious tooling for such implementations include Storm (for event processing), Kafka (for queueing), and ZooKeeper (for tracking and configuration). Such components, written respectively in Clojure (Storm), Scala (Kafka), and Java (ZooKeeper), provide the desired scalability and reliability. But what may not be so obvious at first glance is that we can work with other languages, including Python, for the application level of such architectures. (If so inclined, you can also try reimplementing such components in Python, but why not use something that's been proven to be robust?) In fact Python is likely a better language for the app level, given that it is concise, high level, dynamically typed, and has great libraries. Not to mention fun to write code in! This is especially true when we consider the types of tasks we need to write: they are very much like the data transformations and analyses we would have written of say a standard Unix pipeline. And no one is going to argue that writing such a filter in say Java is fun, concise, or even considerably faster in running time. So let's look at how you might solve such larger problems. Given that it was straightforward to solve a small problem, we might approach as follows. Simply divide up larger problems in small one. For example, perhaps work with one customer at a time. And if failure is an ever present reality, then simply ensure your code retries, just like you might have re-run your pipeline against some input files. Unfortunately both require distributed coordination at scale. And distributed coordination is challenging, especially for real systems, that will break at scale. Just putting a box in your architecture labeled **"ZooKeeper"** doesn't magically solve things, even if ZooKeeper can be a very helpful part of an actual solution. Enter the Storm framework. While Storm certainly doesn't solve all problems in this space, it can support many different types of realtime architectures and works well with Python. In particular, Storm solves two key problems for you. **Partitioning**. Storm lets you partition streams, so you can break down the size of your problem. But if the a node running your code fails, Storm will restart it. Storm also ensures such topology invariants as the number of nodes (spouts and bolts in Storm's lingo) that are running, making it very easy to recover from such failures. This is where the cleverness really begins. What can you do if you can ensure that **all the data** you need for a given continuously updated computation - what is the state of this customer's account? - can be put in **exactly one place**, then flow the supporting data through it over time? We will look at how you can readily use such locality in your own Python code. **Retries**. Storm tracks success and failure of events being processed efficiently through a batching scheme and other cleverness. Your code can then choose to retry as necessary. Although Storm also supports exactly-once event processing semantics, we will focus on the simpler model of at-least-once semantics. This means your code must tolerate retry, or in a word, is idempotent. But this is straightforward. We have often written code like the following: seen = set() for record in stream: k = uniquifier(record) if k not in seen: seen.add(k) process(record)
  • Published: 2014
  • Publisher: EuroPython
  • Language: English
29:12 EuroPython English 2014

Non Sequitur: An exploration of Python's random module

Jair Trejo - Non Sequitur: An exploration of Python's random module An exploration of Python's random module for the curious programmer, this talk will give a little background in statistics and pseudorandom number generation, explain the properties of python's choice of pseudorandom generator and explore through visualizations the different distributions provided by the module. ----- # Audience Non mathematical people who wants a better understanding of Python's random module. # Objectives The audience will understand pseudorandom number generators, the properties of Python's Mersenne Twister and the differences and possible use cases between the distributions provided by the `random` module. # The talk I will start by talking about what randomness means and then about how we try to achieve it in computing through pseudorandom number generators (5 min.) I will give a brief overview of pseudorandom number generation techniques, show how their quality can be assessed and finally talk about Python's Mersenne Twister and why it is a fairly good choice. (10 min.) Finally I will talk about how from randomness we can build generators with interesting probability distributions. I'll compare through visualizations thos provided in Python's `random` module and show examples of when they can be useful in real-life. (10 min.)
  • Published: 2014
  • Publisher: EuroPython
  • Language: English
out of 4 pages
Loading...
Feedback

Timings

  282 ms - page object
  179 ms - search
   16 ms - highlighting
    3 ms - highlighting/19943
    3 ms - highlighting/19964
    4 ms - highlighting/20024
    5 ms - highlighting/19962
    3 ms - highlighting/19971
    2 ms - highlighting/19952
    6 ms - highlighting/20050
    4 ms - highlighting/19939
    1 ms - highlighting/19975
    1 ms - highlighting/19985
    3 ms - highlighting/19981
    3 ms - highlighting/19984
    3 ms - highlighting/19970
    3 ms - highlighting/19983
    2 ms - highlighting/19973
    4 ms - highlighting/19938
    3 ms - highlighting/19966
    2 ms - highlighting/19954
    4 ms - highlighting/20054
    5 ms - highlighting/20043
    3 ms - highlighting/19968
    3 ms - highlighting/19965
    3 ms - highlighting/19969
    3 ms - highlighting/19955
    4 ms - highlighting/20034
    5 ms - highlighting/19961
    3 ms - highlighting/20020
    4 ms - highlighting/19946
    7 ms - highlighting/20051
    4 ms - highlighting/20033
    1 ms - highlighting/19978
    2 ms - highlighting/19951
    2 ms - highlighting/19963
    2 ms - highlighting/19976
    1 ms - highlighting/19974
    2 ms - highlighting/19977

Version

AV-Portal 3.8.2 (0bb840d79881f4e1b2f2d6f66c37060441d4bb2e)