29 Ergebnisse
45:25
5Keith Fiske,New features by demo This all-demo, no-slide talk will show off 9.5's new features. With every new Postgres release comes new features and improvements to make your life easier. Come see some of the new 9.5 features in action and learn how this next release will make your life better.
2015PGCon - PostgreSQL Conference for Users and Developers, Andrea Ross
54:08
196Jim Nasby,The use of Multi-Version Concurrency Control (MVCC) is perhaps one of the most powerful features PostgreSQL has to offer, but it can be a source of confusion for new and experienced users alike. In this talk we will provide an in-depth walkthrough of why Postgres needs to vacuum and what vacuum does. Topics: - MVCC details - HOT overview - Identifying tuples to be vacuumed/frozen - VACUUM and indexes - Vacuuming heap pages
2015PGCon - PostgreSQL Conference for Users and Developers, Andrea Ross
59:57
166Joe Celko,2015PGCon - PostgreSQL Conference for Users and Developers, Andrea Ross
49:10
139Andres Freund,BiDirectional Replication In the course of the BDR (BiDirectional Replication) project we have worked on delivering robust, feature-full and fast asynchronous multi-master replication for postgres. In addition we have started the UDR project, sharing most of the code and infrastructure with BDR, which provides unidirectional logical replication for the many cases where multi-master replication is not required. To implement BDR a lot of features have already been integrated into core PostgreSQL (9.4). Now that 9.4 is released and BDR/UDR is in production in several complex environment there's some important discussions to be had about what can and what cannot be integrated into core PostgreSQL. We will discuss: Which features are in core postgres Which features does BDR/UDR provide on top of that What can be integrated into core PostgreSQL and how Future features Problems found during the development
2015PGCon - PostgreSQL Conference for Users and Developers, Andrea Ross
24:23
8Mello, Fabrízio de RoyesThis presentation is about my experience in the FOSS world and contributing to PostgreSQL as a Google Summer of Code 2014 student. In this presentation I'll talk about all my involvement with the FOSS world and how it change my life and career in many ways. I'll explain how Google Summer of Code works and the importance of this program to the open-source communities. Some points covered: - who can apply - how to apply - how you can help the PostgreSQL community - principal events
2015PGCon - PostgreSQL Conference for Users and Developers, Andrea Ross
1:02:39
338David Steele,PgBackRest is a backup system developed at Resonate and open sourced to address issues around the backup of databases that measure in tens of terabytes. It supports per file checksums, compression, partial/failed backup resume, high-performance parallel transfer, async archiving, tablespaces, expiration, full/differential/incremental, local/remote operation via SSH, hard-linking, and more. PgBackRest is written in Perl and does not depend on rsync or tar but instead performs its own deltas which gives it maximum flexibility. This talk will introduce the features, give sample configurations, and talk about design philosophy. PgBackRest aims to be a simple backup and restore system that can seamlessly scale up to the largest databases and workloads. Instead of relying on traditional backup tools like tar and rsync, PgBackRest implements all backup features internally and features a custom protocol for communicating with remote systems. Removing reliance on tar and rsync allows better solutions to database-specific backup issues. The custom remote protocol limits the types of connections that are required to perform a backup which increases security. Each thread requires only one SSH connection for remote backups. Primary PgBackRest features: Local or remote backup Multi-threaded backup/restore for performance Checksums Safe backups (checks that logs required for consistency are present before backup completes) Full, differential, and incremental backups Backup rotation (and minimum retention rules with optional separate retention for archive) In-stream compression/decompression Archiving and retrieval of logs for replicas/restores built in Async archiving for very busy systems (including space limits) Backup directories are consistent Postgres clusters (when hardlinks are on and compression is off) Tablespace support Restore delta option Restore using timestamp/size or checksum Restore remapping base/tablespaces
2015PGCon - PostgreSQL Conference for Users and Developers, Andrea Ross
49:21
21Hanson, JamesWhy, when and how you can integrate documents and key-value pairs into your relational model There is a pitched battle going on between the relational, document-based, key-value and other data models. PostgreSQL is uniquely capable of leveraging many of the strengths of multiple data models with JSON(b), HSTORE, XML, ltree data types, arrays and related functions. This presentation outlines the use-cases, benefits and limitations of document-based, key-value and hierarchical data models. It then presents practical advice and code snippets for incorporating them into PostgreSQL's relational framework. The presentation ends with SQL examples and code snippets for loading, accessing and modifying (where possible) JSON, HSTORE, XML, ltree and array data types. This presentation begins with a very quick review of the rationale, benefits and implications of the relational data model. It then does the same for document-based models and hierarchical models. The balance of the presentation works with three publicly available data sets, world-wide airports, Wikipedia Inbox key-value pairs and Google address JSON objects, showing how they can be be incorporated into a simple relational model. The presentation also includes snippets of code for loading the files and accessing elements. The full SQL, and shell code will be available on the web site.
2015PGCon - PostgreSQL Conference for Users and Developers, Andrea Ross
45:32
12Sakata, Tetsuo et al.PostgreSQL Enterprise Consortium (PGECons for short) is an organization that consists of major IT companies in Japan, aiming to promote PostgreSQL to enterprise users in the country. Since 2012 when PGECons was established, we have been doing surveys of PostgreSQL's functions and performance to PGECons members to estimate how PostgreSQL well meets their requirements. In this talk, we will focus on some of major requests from the surveys, including enhancement of table partitioning and error messages handling. From the enterprise users' point of view, we would like to share these obstacles behind the requests that might limit PostgreSQL's acceptance, in order to cope with these issues with the community. [Partitioning] More the size of a database grows, the more difficult the design and operation of the system become. For example, a sales accounting application has such difficulty, which is often solved by using horizontal table partitioning. We have surveyed how PostgreSQL can be applied to such kind of applications to the member of PGECons, and evaluated performance of table partitioning. These surveys result suggested PostgreSQL partitioning issues below. larger number of partitions slows query response time definition and usage of partitioned table are intricate From additional surveys about proprietary DBMS usage to the members, we will show what are needed in partitioning and the features of the application areas that are important but difficult to be supported by PostgreSQL. Then, we will point out that enhancing partitioning functions will make PostgreSQL to be spread to the aforementioned areas. [Error messages] When shooting a trouble about PostgreSQL, a user analyzes a trouble to identify the cause based on the error logs and error messages. However, when we analyze a trouble with only SQLSTATE code we sometimes cannot identify the exact cause because an error code is often assigned to multiple different error messages. Also, we have similar difficulty when analyzing a trouble from error messages. To cope with these problems, Fujitsu Limited, a member of PGECons, delivers customers a list of error messages sorted by SQLSTATE codes to search the corrective actions. In the list, an identifier is added to each item so that customers immediately decide corrective actions and communicate smoothly with support staffs. Based on our activities above, we will propose error message systems which will bring following two merits: accelerate decisions on corrective actions enhance the self support by users In this presentation, we will show the needs and actions from members of PGECons as enterprise users, to share the way of improvement, which should help promote PostgreSQL among the community.
2015PGCon - PostgreSQL Conference for Users and Developers, Andrea Ross
51:04
560Bruce Momjian,This talk is for people who want to understand how PostgreSQL shares information among processes using shared memory. The talk covers the internal data page format, usage of the shared buffers, locking methods, and various other shared memory data structures.
2015PGCon - PostgreSQL Conference for Users and Developers, Andrea Ross
46:49
24Samantha Billington,Most of the time, a streaming replication slave in the same data center is so close to the master that lag can be measured in milliseconds. However when it's not, that lag can be baffling at best, and catastrophic at worst. We will look at all things lag; strategies of monitoring, configuration options to fit application needs, diagnosing common issues and real cases of 'what went wrong'. If you google from "postgres streaming replication lag" (go ahead, I'll wait...) your result set will include much information on set up and monitoring, but very little on diagnosing and even less on correcting. This talk is an attempt to fill that gap. We will start with the basics of monitoring and trending over time, look at configuration options and 'gotchas' for making your slaves trusted read sources, diagnose hardware and system factors, and finally share the pain of elusive lag patterns that took days, if not weeks to figure out. This talk takes a broad look at system health. Many factors contribute to making a database cluster run perfectly; disk speed, network latency, user query patterns, etc., etc. It can be easy to over look, or take for granted things that may strongly effect how close a slave follows the master. In fall of 2014 iParadigms converted 8 server clusters across two data centers to streaming replication, allowing us to find and document many such issues.
2015PGCon - PostgreSQL Conference for Users and Developers, Andrea Ross
49:46
59Jeremy Smith,Keeping track of changes in your database schema can be challenging. In this talk I will discuss the advantages of using Flyway to effectively manage this issue. Migrations are an essential tool for both developers and administrators. Developers can quickly recreate a database from scratch and incrementally modify their development database along with their code and tests. Similarly, administrators can determine the current state of any database and easily migrate to a newer one. Most importantly, schema and data changes can be thoroughly reviewed and tested before going to production. In this talk, I will discuss the benefits of using Flyway to manage migrations. Specifically, I will: - Show why migrations are useful - Introduce Flyway and how to use it - Focus on using Flyway from the command line using migrations written in sql - Help you determine which changes should be in your migrations - Discuss how to create a base migration from your existing database - Cover strategies for dealing with global objects - Show how to integrate Flyway with Jenkins
2015PGCon - PostgreSQL Conference for Users and Developers, Andrea Ross
53:03
187Markus Winand,A lot has changed since SQL:92 SQL has gone out of fashion lately --- partly due to the NoSQL movement, but mostly because SQL is often still used like 20 years ago. As a matter of fact, the SQL standard continued to evolve during the past decades resulting in the current release of 2011. In this session, we will go through the most important additions since the widely known SQL-92, explain how they work and how PostgreSQL extends them. We will cover common table expressions and window functions in detail and have a very short look at the temporal features of SQL:2011 and the related features of PostgreSQL.
2015PGCon - PostgreSQL Conference for Users and Developers, Andrea Ross
50:57
42Horiguchi, Kyotaropg_statsinfo comes with new features There are many information vanishes as the server operates. pgstatsinfo/pgstatsreporter is a monitoring tool which records such various status and statistics of PostgreSQL server and lets you see them in graphical and interactive way. It is very usable not only for DBAs to check the health of the server daily, but for technical support to find out what happened in the past on the remote site. It is widely adopted among our systems using PostgreSQL and the new pgstatsinfo 3.1 has new features to support them more. The new pg_statsinfo 3.1 has the following new features in comparison to 2.5. This talk will introduce these features with demo and dig inside some of them. Collecting plan statistics. Plan statistics is based on an original pgstatstatements-like extension named pgstoreplans, which is a similar tool to 2ndQuadrant's pgstatplans but it still differs in some points to fit to pg_stastinfo. Statistics of autovacuum/analyze including cancellation stats. Cancellation stats would be in some situations. Storing server logs into the repositiry database. Stored logs are examined using filtering feature of pgstatsreporter. Storing alerts previously only emitted into server log. Alerts gets more valuable with pgstatsreporter.
2015PGCon - PostgreSQL Conference for Users and Developers, Andrea Ross
49:12
274Shaji, ArulThis talk is about the need of multi-tenancy in PostgreSQL, and the way to achieve multi-tenancy in PostgreSQL. What is a multi-tenant cluster? Why multi-tenant cluster is needed? PostgreSQL Provides multi-tenancy with the following - Shared Database, Shared Namespace - Separate Databases - Shared Database, Separate Namespace However multi-tenancy means more than this. - Issues with a multi-tenant cluster - What can be done and what can we do to make it easier. This talk will propose a multi tenanted architecture for PostgreSQL, to make it the database of choice in a cloud environment. Multi tenanted architecture is one of the key requirements for any software to be efficiently deployed in the cloud. As more and more databases are made available 'as-a-Service' in cloud offerings, it is necessary to take stock of the features in PostgreSQL to analyse how cloud friendly they are, especially for a multi-tenanted infrastructure. This talk will mainly focus on what functionalities are needed in PostgreSQL to make it truly cloud friendly. PostgreSQL needs to have the functionalities that will make it the database of choice for service providers in the cloud. This can be achieved within the current architecture of PostgreSQL by developing new features that will satisfy these requirements.
2015PGCon - PostgreSQL Conference for Users and Developers, Andrea Ross
57:02
33Haas, Robert et al.Unleashing a heard of elephants Parallel query is close to becoming a reality in PostgreSQL! A year ago, much of the low-level infrastructure needed for parallelism, such as dynamic shared memory and dynamic background workers, had been completed, but no user-visible facilities made use of this infrastructure. Major work on error handling, transaction control, and state sharing has been completed, and further patches, including a patch for parallel sequential scan, are pending. In this talk, we will talk about parallel sequential scan itself, including performance considerations, the work allocation strategy, and the cost model; and we will also discuss the infrastructure that supports parallel sequential scan, including state sharing for GUCs, transaction state, snapshots, and combo CIDs; error handling and transaction management; and the handling of heavyweight locking. Finally, we'll discuss the future of parallelism in PostgreSQL now that the basic infrastructure is (mostly) complete.
2015PGCon - PostgreSQL Conference for Users and Developers, Andrea Ross
49:46
201Erdogan, Ozgun et al.PostgreSQL extension to scale out real-time reads and writes pg_shard is an open source sharding extension for PostgreSQL. It shards PostgreSQL tables for horizontal scale, and replicates them for high availability. The extension also seamlessly distributes SQL statements, without requiring any changes to the application layer. pg_shard addresses many NoSQL use-cases, and becomes more powerful with the new JSONB data type. Further, the extension leverages the rich analytic capabilities in PostgreSQL, and enables real-time analytics for big data sets. In this talk, we first summarize challenges in distributed systems associated with scaling out databases. We then describe "logical sharding", and discuss how it helps overcome these challenges. Next, we show how pg_shard uses hook APIs, such as the planner and executor hooks, to make PostgreSQL a powerful distributed database. We then cover example customer use-cases, and conclude with a futuristic demo: a distributed table with JSONB fields, backed by a dynamically changing row and columnar store. pg_shard is an open source sharding extension for PostgreSQL. It shards PostgreSQL tables for horizontal scale, and replicates them for high availability. The extension also seamlessly distributes SQL statements, without requiring any changes to the application layer. pg_shard addresses many NoSQL use-cases, and becomes more powerful with the new JSONB data type. Further, the extension leverages the rich analytic capabilities in PostgreSQL, and enables real-time analytics for big data sets. In this talk, we first summarize challenges in distributed systems: dynamically scaling a cluster when new machines are added or old ones fail, and distributed consistency semantics in the face of failures. We then describe "logical sharding", and show how it helps overcome these challenges. We also discuss this idea's application to Postgres. Next, we show how pg_shard uses hook APIs, such as the planner and executor hooks, to make PostgreSQL a powerful distributed database. We then cover example customer use-cases, and conclude with a futuristic demo: a distributed table with JSONB fields, backed by a dynamically changing row and columnar store.
2015PGCon - PostgreSQL Conference for Users and Developers, Andrea Ross
49:29
19Korotkov, Alexander et al.PostgreSQL 9.4 has introduced JSONB, a structured format for storing JSON, which provides many users with the new opportunity: an effective storing and querying JSON documents inside ACID relational database. While users have notice a great jsonb performance, their feedback also reveals some hidden problems with current jsonb implementation. We want to discuss different approaches to resolve aforementioned problems and present several proof-of-conceps, so we could rethink jsonb for 9.6.
2015PGCon - PostgreSQL Conference for Users and Developers, Andrea Ross
50:07
13Frost, StephenIn this talk we'll review Row-Level Security (RLS), provide examples and use-cases, discuss the work which has been done on adding Row Level Security to PostgreSQL and the current state of that effort. PostgreSQL has long had a complex and interesting set of permissions available through the GRANT system. There is another system which exists in many other RDBMS's known as row-level security (RLS), where the rows returned is filtered based on a policy implemented on the table.
2015PGCon - PostgreSQL Conference for Users and Developers, Andrea Ross
40:44
44Kapila, AmitThis paper will main talk about the scalability and performance improvements done in PostgreSQL 9.5 and will discuss about the improvements that can be done to improve the scalability for both Write and Read operations. The paper will focus on pain points of Buffer Management in PostgreSQL and the improvements done in 9.5 to improve the situation along with performance data. It will also describe in brief the performance improvements done in 9.5. It will also discuss the locking bottlenecks due to various locks (lightweight locks and spinlocks) taken during Read operation and what could be done to further scale the Read operation. The other part of the paper focusses on improving the Write-workload in PostgreSQL. In this part we will discuss about the frequency of writes done by backend operations (along with data) due to limitations of current bgwriter algorithm and some ideas to improve the performance by reducing writes done by backend. It will also discuss about the concurrency bottlenecks in write operation and some ideas to mitigate the same.
2015PGCon - PostgreSQL Conference for Users and Developers, Andrea Ross
39:51
14Kumar, Dilip et al.In Current MVCC solution of PG, ProcArrayLock is the major bottleneck on many core machine 120+ and can scale up to 30 connections in TPCC test. Done experiment with lock free MVCC solution, and it can Scale up to 120 cores. In Current MVCC solution of PG, ProcArrayLock is the major bottleneck on many core machine 120+ and can scale up to 30 connections in TPCC test. Done experiment with lock free MVCC solution, and it can Scale up to 120 cores. We have taken the CSN based solution proposed in PG community, and implemented a lock free version of the same. By considering the High Memory and other resources in many core machines, locks are avoided in all the performance patch and only in some rare paths locks are used.
2015PGCon - PostgreSQL Conference for Users and Developers, Andrea Ross
40:00
17McDougall, Joshua1v1 Schemaverse Battles A tournament of 1v1 schemaverse battles, each round taking only about 10 minutes. No prepared scripts allowed! The Schemaverse is a space-based strategy game implemented entirely within a PostgreSQL database. Compete against other players using raw SQL commands to command your fleet. Or, if your PL/pgSQL-foo is strong, wield it to write AI and have your fleet command itself! This year, rather than the classic large space battle, the rounds will be 1v1 and only take 10 minutes each. There will also be NO pre-created scripts allowed. Matches will be broadcast live for all to see and a ladder updated after each round.
2015PGCon - PostgreSQL Conference for Users and Developers, Andrea Ross
44:23
8Conway, JoeScripting with Postgres Sometimes bash is just the way to go! This talk will cover tips and techniques for effective bash scripting with PostgreSQL. Sometimes bash is just the way to go! This talk will cover tips and techniques for effective bash scripting with PostgreSQL. It will include guidance about: Pros/cons of shell scripts Function library creation and use Executing SQL Set/get PostgreSQL data from/into script variables Keeping PostgreSQL functions in sync with scripts Locking Doing work in parallel Ensuring cleanup This is a source-code heavy talk. Moderate experience with both bash scripting and PostgreSQL is needed to get the most out of it.
2015PGCon - PostgreSQL Conference for Users and Developers, Andrea Ross
54:10
5Berkus, JoshShootout at the PAAS Corral head-to-head for PostgreSQL cloud platforms Where should you run your PostgreSQL in the cloud? Join us for a comparison of features, pricing and performance between various cloud options, including most or all of EC2, Amazon RDS, Heroku, OpenShift, Google Compute, and the Rackspace Cloud. To determine which cloud is the fastest, cheapest and best, over the next few months Josh Berkus and others will be running a series of performance benchmarks against several of the many cloud hosting options available for PostgreSQL. This will include most or all of EC2, Amazon RDS, Heroku, OpenShift, Google Compute, and the Rackspace Cloud. The results will be presented to you in this talk, including: Benchmarking methdology Cost comparison for each configuration Feature differences Performance scores
2015PGCon - PostgreSQL Conference for Users and Developers, Andrea Ross
38:45
23Hayamizu, Yuto"Contrary to common belief, performance evaluation is an art." (Raj Jain, 1991) Successful performance evaluation may not be achieved with merely executing common benchmarking tools. This talk presents fundamental principles of performance evaluation and how you can put them into practice. Do you understand what exactly "pgbench" does? Is it appropriate workload for your performance evaluation goal? Common benchmarking tools like "pgbench" are handy for just comparing system A and system B, but if you intend to deeply understand the performance of your system, answers to these questions are critical. In order to conduct a meaningful performance evaluation, the methodology should be elegantly designed to meet the goal of the evaluation: choose metrics for the goal, and choose observation techniques for the metrics. Each step requires careful consideration and deep knowledge about the target system. It cannot be done mechanically. This is why performance evaluation is an art. This talk presents principles of designing performance evaluations and shows how you can put them into practice by introducing the speaker's experiences of performance evaluations with PostgreSQL.
2015PGCon - PostgreSQL Conference for Users and Developers, Andrea Ross
43:45
81Kosmodemiansky, Ilya2015PGCon - PostgreSQL Conference for Users and Developers, Andrea Ross
50:48
10Bapat, AshutoshManaging transactions involving multiple foreign servers. PostgreSQL has Foreign Data Wrappers and they are writable too! Upcoming features like partitioning, foreign table inheritance and join-push down for foreign tables pave the path for sharding. One missing piece in the puzzle is the distributed transaction manager required to maintain the atomicity and consistency of transactions involving foreign servers. The presentation talks about the current status of such transactions and discusses the path forward towards distributed transaction manager. Support for writable foreign tables was added in PostgreSQL 9.3. As of now, atomicity and consistency is guaranteed, when a transaction makes changes to at most a single foreign server. It fails to do so when changes are made to multiple foreign servers. In order to achieve atomicity and consistency of transactions involving multiple foreign servers, PostgreSQL needs to take up the role of a distributed transaction manager. The talk covers the current status of distributed transactions. It further explores protocol to drive distributed transactions and infrastructure necessary to overcome various hardware, software and network failures during a distributed transaction. It also covers the use cases like data federation and sharding.
2015PGCon - PostgreSQL Conference for Users and Developers, Andrea Ross
16:51
413Dunstan, Andrew et al.Postgres 9.4 introduced the new jsonb type, However, it is missing some functions, particularly for json composition, that are needed by many users. In this talk we present an extension that provides some of these functions, and work to incorporate the functions in 9.5. Operations include replacement and deletion of array elements and object fields, and composition by concatentation of objects and arrays. Arrays can also be concatenated with scalar values, and array elements can be replaced or deleted by counting from either end of the array. Thus we have the ability to use json arrays as queues and stacks, with basic push/pop and shift/unshift capability. A pretty print function for jsonb is also provided. We will also outline what work we think remains, and discuss possible ideas on how to make json composition more naturally expressed.
2015PGCon - PostgreSQL Conference for Users and Developers, Andrea Ross
25:36
56Dunstan, Andrewproviding some needed functions and operators for jsonb Postgres 9.4 introduced the new jsonb type, However, it is missing some functions, particularly for json composition, that are needed by many users. In this talk we present an extension that provides some of these functions, and work to incorporate the functions in 9.5. Operations include replacement and deletion of array elements and object fields, and composition by concatentation of objects and arrays. Arrays can also be concatenated with scalar values, and array elements can be replaced or deleted by counting from either end of the array. Thus we have the ability to use json arrays as queues and stacks, with basic push/pop and shift/unshift capability. A pretty print function for jsonb is also provided. We will also outline what work we think remains, and discuss possible ideas on how to make json composition more naturally expressed.
2015PGCon - PostgreSQL Conference for Users and Developers, Andrea Ross
48:14
44Linnakangas, HeikkiPeople has been setting up warm standby systems with streaming replication since version 9.0, and even longer with file-based log-shipping. However, there has been a few pitfalls that many people don't know about, while others have simply accepted the risks. PostgreSQL 9.5 brings a bunch of new features and subtle changes that make warm standby setups more robust than ever. In 9.5, the interaction between a WAL archive and failover has been revised. pg_rewind makes it possible to resynchronize an old master server after failover - even an unplanned one. Replication slots, already introduced in 9.4, make the behaviour of a standby falling behind nicer. This presentation explains the changes, and why they were needed. Finally, I'm going to walk through setting up a simple, robust, two server hot standby system, using only built-in tools and simple shell scripts, taking advantage of the new features.
2015PGCon - PostgreSQL Conference for Users and Developers, Andrea Ross