Understanding and Applying CQRS
This is a modal window.
The media could not be loaded, either because the server or network failed or because the format is not supported.
Formal Metadata
| Title |
| |
| Title of Series | ||
| Number of Parts | 132 | |
| Author | ||
| License | CC Attribution - NonCommercial - ShareAlike 3.0 Unported: You are free to use, adapt and copy, distribute and transmit the work or content in adapted or unchanged form for any legal and non-commercial purpose as long as the work is attributed to the author in the manner specified by the author or licensor and the work or content is shared also in adapted form only under the conditions of this | |
| Identifiers | 10.5446/44873 (DOI) | |
| Publisher | ||
| Release Date | ||
| Language |
Content Metadata
| Subject Area | ||
| Genre | ||
| Abstract |
|
EuroPython 2018132 / 132
2
3
7
8
10
14
15
19
22
27
29
30
31
34
35
41
44
54
55
56
58
59
61
66
74
77
78
80
81
85
87
91
93
96
98
103
104
105
109
110
111
113
115
116
118
120
121
122
123
125
127
128
129
130
131
132
00:00
Moment (mathematics)Multiplication signSoftware engineeringComputer animation
01:08
Information managementArtificial lifeMereologyMoving averageTwitterTouchscreenLink (knot theory)Information securityE-bookContent (media)Rule of inferenceComputer animation
01:52
Query languageObject (grammar)Pattern languageObject (grammar)Server (computing)Dependent and independent variablesQuery languageComputer animation
02:50
ArchitectureRepository (publishing)Time domainData modelHill differential equationInstance (computer science)Computer architectureNormal (geometry)Cartesian coordinate systemData storage deviceInstance (computer science)Domain nameLimit (category theory)Bootstrap aggregatingClient (computing)Game controllerRepository (publishing)Virtual machineProcess (computing)WordComputer configurationScalabilityEndliche ModelltheorieScaling (geometry)Moment (mathematics)Reading (process)2 (number)InternetworkingComputer fileLinearizationComputer animation
05:21
Repository (publishing)Time domainData modelWritingCartesian coordinate systemComputer configurationSinc functionGoodness of fitQuery languageConsistencyProcess (computing)Event horizonMathematicsComputer animation
07:05
Cache (computing)Repository (publishing)Time domainData modelData storage deviceQuery languageComplex (psychology)Computer configurationRight angleGame controllerSemiconductor memoryMetropolitan area networkCache (computing)Strategy gameNumberQuery languagePattern languageReading (process)CodeWritingSocial classProcess (computing)SynchronizationConsistencyBit error rateSet (mathematics)FacebookDatabaseMultiplication signData storage deviceComputer animation
10:02
InformationQuery languageOperations researchStack (abstract data type)State of matterQuantum stateQuery languageStack (abstract data type)Dependent and independent variablesProcess (computing)Goodness of fitComputer animation
10:57
Stack (abstract data type)Query languageRenormalizationCurvatureQuery languageSubject indexingCurveNumberLevel (video gaming)Factory (trading post)DatabaseComputer animation
11:51
Stack (abstract data type)Event horizonClient (computing)DatabaseStack (abstract data type)ResultantConnectivity (graph theory)Imperative programmingMereologyCASE <Informatik>Order (biology)Data storage deviceEvent horizonQuery languageSynchronizationComputer animation
13:01
SynchronizationStack (abstract data type)Query languageEvent horizonProcess (computing)SynchronizationQuery languageInformationBinary codeConnectivity (graph theory)Graphical user interfaceDiagramEvent horizonMultiplication signGame controllerWordComputer configurationData storage deviceCentralizer and normalizerService-oriented architectureBus (computing)Queue (abstract data type)CurveStack (abstract data type)Process (computing)TelecommunicationComputer animation
15:02
DiagramPoint (geometry)BitService (economics)Query languageInformationEvent horizonCycle (graph theory)Figurate numberDomain nameSynchronizationConnectivity (graph theory)Multiplication signNormal (geometry)Client (computing)InternetworkingComputer animationEngineering drawing
17:07
Video gameCodeArtificial lifeSoftware frameworkRepetitionHorizonProjective planeLink (knot theory)Computer animation
17:47
Event horizonService (economics)Multiplication signDiagramConnectivity (graph theory)Computer animation
18:28
Open setSummierbarkeitFinite element methodMenu (computing)QuantumService (economics)Figurate numberCodeComputer animation
19:16
Price indexEmailOpen setLine (geometry)Social classQuery languageElectronic mailing listAdditionQuadrilateralEndliche ModelltheorieEvent horizonModemComputer wormStructural loadWeb pageDependent and independent variablesLimit (category theory)System on a chipExecution unitLinear partial informationPermianContent (media)Rollback (data management)Maxima and minimaData modelInfinite conjugacy class propertyDean numberObject (grammar)Normed vector spaceQueue (abstract data type)Domain nameSoftware frameworkComputer wormDifferent (Kate Ryan album)Service (economics)Query languageStack (abstract data type)Event horizonMethodenbankConnectivity (graph theory)Point (geometry)AdditionInstance (computer science)CodeElectronic mailing listDatabaseSynchronizationTelecommunicationComputer fileUniform resource locatorEvent-driven programmingEndliche ModelltheorieConfiguration spaceQueue (abstract data type)Metropolitan area networkSystem administratorData structureOrder (biology)InformationService-oriented architecture2 (number)Repository (publishing)Validity (statistics)Centralizer and normalizerPatch (Unix)Source codeXMLComputer animation
24:46
Line (geometry)Queue (abstract data type)Query languageModemMenu (computing)Core dumpExecution unitEmailSample (statistics)Sign (mathematics)Uniform resource locatorInformationDomain nameInsertion lossSystem administratorSource codeComputer animation
26:18
Lemma (mathematics)Cartesian coordinate systemSystem administratorDescriptive statisticsQuery languageUsabilityLattice (order)Point (geometry)Video gameStack (abstract data type)AuthorizationArithmetic meanComputer animation
27:52
Process (computing)Traffic reportingCartesian coordinate systemData storage deviceSoftware testingStack (abstract data type)Moment (mathematics)Service (economics)Query languageQueue (abstract data type)MiniDiscCentralizer and normalizerComputer animationProgram flowchart
29:42
Event horizonConsistencyMessage passingQueue (abstract data type)ArchitecturePattern languageEvent horizonComputer architectureStability theoryMessage passingLoginMoment (mathematics)ScalabilityConsistencyMultiplication signCycle (graph theory)Service-oriented architectureJava appletCartesian coordinate systemGoodness of fitCategory of beingSynchronizationMusical ensembleInformation securitySoftware bugImplementationComputer configurationLevel (video gaming)Queue (abstract data type)Computer animation
33:48
Social classPattern languageComputer-assisted translationTelecommunicationService (economics)Link (knot theory)Computer animation
34:49
DatabaseInstance (computer science)MereologyPattern languageStatisticsRevision controlDifferent (Kate Ryan album)Scaling (geometry)Stack (abstract data type)Query languageQueue (abstract data type)BitData storage deviceOrder (biology)Right angleSineSource codeDependent and independent variablesGame controllerOperator (mathematics)Human migrationMathematicsSet (mathematics)Process (computing)CurvaturePlastikkarteScalabilityMathematical optimization2 (number)Cache (computing)WritingBlock (periodic table)Computer animationMeeting/Interview
40:14
LoginDatabaseSemaphore lineInformationStack (abstract data type)Bus (computing)MereologyEvent horizonElement (mathematics)Right angleStreaming mediaLogicRelational databaseSign (mathematics)Process (computing)Representation (politics)Service (economics)Complex (psychology)Cartesian coordinate systemSource codeImplementationMultiplication signDifferent (Kate Ryan album)Query languageEndliche ModelltheorieData streamStrategy gameWeb 2.0Server (computing)Lecture/Conference
45:20
MassEmailPrice indexSocial classHeegaard splittingQuery languageDomain nameLogicInformationEvent horizonRule of inferenceConnectivity (graph theory)Table (information)Normal (geometry)Computer animationXML
46:12
TorusCAN busComputer wormDuality (mathematics)Rollback (data management)EmailGoodness of fitEvent horizonDomain nameConnectivity (graph theory)LogicMereologyRepository (publishing)Computer animationXML
47:20
Maxima and minimaPlastikkarteCAN busComputer animation
Transcript: English(auto-generated)
00:05
Thank you. I'm Venezio Pacheco. I'm from Brazil. I have a funny accent. I'm based on Argentina. I'm a senior software engineer at Eventbrite. It's a very cool company.
00:27
Without the support from this company, it's impossible to be here. I'm coming from Mendoza. It was 20 hours in flight. So, Mendoza, Argentina. Please, applause for this company. They are amazing. Thank you.
00:46
If you don't know me, every time I give normally, Star Wars talk. And today, it's not different. So, after this moment, I can't hear you.
01:03
Okay. Now, it's better. I released a book earlier this year, in January. And the content of this talk, the subject of this talk, make part of my book. There's the link. And my Twitter, too, if you want to buy it.
01:23
The e-book's very cheap. I think that's just £9, something like this. And let's start. Our agenda. What is CQRS? When can we apply this? Understanding CQRS, applying, there is a live code, and the common mistakes.
01:43
When you see a Yoda on the screen, there is a live code, okay? It's the rule. So, what is CQRS? Someone here has been working with CQRS. Just to know. Well, a couple of people. Someone know what is CQRS,
02:06
or listen something about this pattern? Okay, great. So, there's no doubt. Command, query, responsibility, segregation. It's what means CQRS.
02:25
It was a pattern documented by Greg Young, and he told this about the pattern in 2010. Basically, where you have just one object responsible to change things in the server, and to retrieve data in the server, you will split this in two different segments.
02:45
That one will be responsible to write, and another one responsible to read. So, when can we use CQRS? There is Chewbacca. I love Chewbacca.
03:00
He's fine. So, the normal architecture. Thinking the normal application architecture. No, no, no. It's not this architecture. This one. There is a UI, or API, a domain, a repository, a model, that's responsible to connect with the DB, and to write, and to read stuff, and could have a controller or something like this.
03:26
It is our bootstrap application. So, it's an instance of my bootstrap application, and it's connected with my storage. And when I deliver a business to the internet,
03:41
there is a word that's very dangerous to my business. This word is success. And why? With success, there are more customers, more clients, more people connecting with me, and now I have a problem, and why? My application has a limit, and I will rent a new machine.
04:04
Okay. It's great. And this process is repeated over and over and over again, and I have a new instance, and another instance, and now I could figure out that my scalability is not more linear.
04:21
My second instance could have a linear scalability, like, okay, I have, I don't know, 10,000 requests per second, and now I have 20,000 requests per second, but there is a moment that my scalability is not the same.
04:40
I cannot scale 50% or just 10% for each new instance. And why? Because maybe your issue is not in your application, but in the storage. And there is another instance that doesn't scale very well, and now it's over.
05:01
My storage starts to create too much latency, or he's very responsible to do things that he's not prepared to do. And my business is out. It's a sadly day.
05:21
So, there is a first option to try to scale to solve my problem, and do you know what is this option? Someone knows? Someone? No, no? A master slave. It is my first option. There is a slave, you know, like a slave from Jabba the Rutt,
05:43
and yeah, she's a slave, yeah, okay. Master slave. And there is my application. Now I have the master slave, and I'm just replicating the data from my master, and my master is going to receive all the writing process, and I'm going to replicate this to my slave
06:03
that will be responsible to retrieve the data, but in fact, I'm not optimizing the data. I'm not optimizing the queries. I'm just replicating the data, and it's very hard to measure how many slaves I have, and when my problem is in the writing process, my problem is to write, it's very difficult to figure out
06:26
how I could improve stuff on master, and it's not the solution for this specific kind of issue. Okay. It's a good approach, but not a solution, and there is another stuff, eventual consistency.
06:41
If you are not, if you don't have the good conscience that, okay, there's in my mind, I would like to use eventual consistency. Okay, it's okay, eventual consistency is good sometimes, but if you don't have this in your mind that accidentally you are using eventual consistency,
07:01
since it's an issue for you. Okay. And there is a second option. Do you know what is this option? Someone knows? Someone knows? Mind, mind to remember like control. No, it's close, but it's not command.
07:23
Someone, someone? No? Okay. It's cache. Because it's memory, you know, like my trick. So, yes, raise your mind. So, it's a good strategy. Now, I'm put to cache, and it will remove some
07:44
heavy processes over my database, and it's okay. But I'm not optimising the queries yet, and I'm optimising a segment that normally is not so expensive than the writing process.
08:02
So, read normally, usually, is easier than write things in DDB. And I'm trying to optimise my issue that's write and read, just optimising the read process. Okay, and there is another stuff
08:21
that's very hard. If you are old in IT, you know that there are two things that are very difficult. The first one is name it classes, and the second one is invalid cache. So, and now I have data synchronisation. The first one is invalid cache? Okay.
08:43
So, it was a pleasure, but I can't hear you. Sorry. It's there yet. When CQRS is OTO, when my storage is bottleneck, definitely, you could use this pattern as a solution. When I have complex queries to my ORM, and they could optimise these queries.
09:05
And when I have a big number of users updating a small data set, and the data could be outdated. In fact, if you think on the Facebook feed, or in the Instagram, every time you are consuming
09:22
data that is outdated. And why? Because when you are looking, someone updates the feed. So, it's eventual consistency every time. It's not, okay, I have my consistency, it's very well, it's totally right. No, no, no, no, it's not true. It's not true.
09:44
In fact, for business that a huge number of users are updating a small data set, or a small piece of code, like a feed, you have eventual consistency every time. And for this, CQRS is a good pattern.
10:03
So, understanding CQRS. Teach me, Sadre. I don't like the last two movies. But, yeah, I prefer, Star Wars has just six movies. And the cartoons, I like the cartoons, colonial wars. Come back.
10:28
Yeah. Oh, look, Jar Jar Binks will appear here. Yeah, yeah, you will see, you will see. You will see. Not boo yet. There is a good motivation.
10:40
Query stack, in the common stack. Basically, the CQRS is divided in two big pieces. The query stack that's responsible for data and the command stack that's responsible to demonstrate my writing processes. Okay. The query stack is simple.
11:01
than the command stack, is synchronous normally, and presumes flat queries. Do you know what is a flat query? Someone don't know what is a flat query? Okay. A query could have joins. It's a simple explanation about flat queries, okay? It's a poor explanation
11:24
about flat queries. A flat query is every query that I'm going to optimise the data to reduce the number of joins that I'm going to do in my database. So, if I know which kind of data I'm going to retrieve, I will prepare this data to return with just one index or two index, but not doing joins and going to the deep level on my data. So,
11:48
it's basically this. And the command stack is potentially asynchronous. It has a behaviour centric. Each command looks like a use case, and has imperative fashion, so it's not more
12:08
just a post for my end point user. No, the end point is an imperative command like create user, or update the user name, or something like this. It's a command. And the command
12:24
handlers just return success or failure, like I accept your request, but I don't know what happened. I don't know what will be the result. And the command updates the own entity that make parts of the command stack, and going to dispatch events that
12:45
there is an event component that will be responsible to update or to normalise the order DB, or the order storage that is responsible with the query stack. Synchronisation.
13:04
There are four ways to synchronise data using CQRS that's like by the book. So, the first one is the automatic update. The automatic update is like I receive the data from my put process, my post or put, my command, and now I'm going to send an
13:26
event that is totally synchronous using RPC, or binary communication, or REST, and it doesn't matter, but the component responsible to normalise the DBs will do this stuff
13:44
instantaneously. So, there is the update possible. Update possible is the most common synchronisation in CQRS. So, the command going to dispatch an event, and the storage is
14:02
information on a bus, on a broker, on a queue, and the component will be responsible to consume this event and normalise the DB on the query stack. But there is all the time of the word to do this. The third option is the control update. So, I go into
14:22
update to normalise my DBs every two minutes is an example. There is a Chrome responsible to normalise the data. And the other option is the update on demand. So, when I receive
14:42
a get to my query stack, the query stack is going to check, hey, I'm out-dated. If I'm out-dated, I'm going to say, hey, please, come on, give me the data, and they're going to be created a new event that the query stack is going to consume and write on DB. So, queuing. It's a diagram that there are many, many, many diagrams
15:09
about CQRS, and I like specifically this one, and why? Because it's like if you follow this diagram, you could use all the synchronisation styles, okay? And it's a very
15:28
flexible diagram. So, there is a client UA API internet that's going to talk with my public facing if we are thinking services, and my public facing going to deliver a
15:44
command. The command will enqueue the information, and my domain layer gets the information, and the domain knows what is necessary to do, and the command will update my normalised DB. Normally, this one here is a SQL DB, and going to
16:09
dispatch an event, this event will be enqueued, and a component, an event component will start to consume the skill, and write this stuff in the
16:23
command. So, the command is unnormalised DB with time, or directly, but normally with time, usually with time, and this unnormalised DB is very common to see no SQLs here, or document DBs. It's common. It's not mandatory, okay? But the main
16:51
point is this command will create the flat query, so optimise the stuff here to be
17:01
defined easily by this request. So, let's figure out something here. Applying CQRS. If there is a Yoda, there is a live code. All the code is on this link. There is a
17:31
link. I'm going to download this, and do some experiments. I'm using the Nameco framework in this example, and Docker, Mongo, PulseGreece, and ... so, it's the same
17:49
diagram, but now I change the names of the components to be clear what is going to happen in our service. It is a user service, okay? So, let's see how long time I have.
18:06
So, my create user, it's the imperative command, and there is a convention that every event we will name it in the past, so user created. If I dispatch another event,
18:25
you'll be in the past too. So, let's see. No, let's come back. Now, the code. Let's figure out what we have here. There is my Docker Compose. The first stack, the DB
18:41
is a MongoDB here. It's a MongoDB. The command stack is PulseGreece. There is my command stack here, okay? My PulseGreece, and I'm using SQL, I read SQL, okay? And
19:11
my user service. The user service entities are the user model, okay, and my permissions
19:26
model. I could have two permissions, admin or user. So, I'm going to do a post, and I will write a new user, and the permission that the user has, and that's it. There
19:47
is the other entities or structures that I'm going to use in my MongoDB. Okay, the service. I could write this in different modules like domain module, repository module,
20:05
these kind of things, but I wrote all the code in the same file to be easy to navigate. Okay, there is the framework and some configurations, but it's my API, okay? My API.
20:21
There is a reference to my query stack, and it is my post. My post is the first layer on my command stack. This post going to receive the data from my request, validate if the payload is a valid payload, and prepare a cluster of RPC communication to do
20:46
a sync RPC, and to put this on my queue, and I create a location, and I go into this patch and 202 event with the location where will be the data,
21:04
when the data obviously going to be consumed by my query stack. Okay, and my API, there are an API to get all users paginated, and the user by ID, and the user by permission, so I'm going to get, I want to get all the users
21:27
that has the same permission. It could be a join on my DB because I have two different entities, but won't be. It will be a flat query. So there is my command stack, and inside my
21:44
command stack I put my user domain. It could be another module domain, but now it's there. So, my command stack, I have an, it's just to use the framework, it's a dispatcher, an instance of even dispatcher, and an instance of the DB. Now I create my entity,
22:06
I commit this, and I dispatch an event that's user-created with the data, and I will dispatch another event to create the relationship between permissions and users. That is my permission user-related, okay? Could be a broker, so I could have two different event components
22:29
listing the same event using a broker, but I'm dispatching two events just to be easy to look what happened. And there is my event component. So, look at this, where we are stopping. The user,
22:47
queue, the domain, you dispatch user-created, and now I'm going to show the component. The component is going to listen the user-created event,
23:01
and get this stuff, create the data, and normalise the data on my MongoDB. The other
23:20
thing is I'm going to create a user structure, look if my permission was created or not. If not, I'm going to create one. If it was created, I will just append this user structure on the permission that's existing on my DB. So, in this event, on this event, I'm optimising
23:41
the data. And why? Because I removed the possibility to use joins, because all my data was prepared to be retrieved with just one ID, that is the admin ID, or the user, the permission ID, sorry. And now, the query stack. The query stack, there are just endpoints using RPC to communicate
24:07
with my API. The first endpoint gets the information from the MongoDB, the second endpoint gets paginated information about users, and my get user permissions, my permissions, is
24:23
exactly that going to find on the flat query, and if you take a look, it's the same query, the same style of query, that I'm using to get a user by ID. So, it's very easy. It's
24:46
let's see if my containers are running. Oh, it's fine. And let's start to consume, to
25:01
send the data. So, first, I'm going to get the info. There's nothing. No users. And now, I'm going to insert the first user, that's me. It's just a simple test, and this user is an
25:26
admin. Okay. Sent it. There is. He was accepted. Okay. I don't know what happened. And there is a location with my data. Okay. And now, if I find here,
25:46
there is my user. Okay? I'm going to create another two users. The other one is, that's me again, and it's an admin, was accepted, and now, I'm going to create the, but now,
26:05
as a user. I don't remember if it was accepted. Let's see. This was too fast to me.
26:21
Oh, no, it was not the last. Now, so, if I find here, there are three users. Admin, admin, and user, but I have the user description, but not the description user. So,
26:45
now, I'm going to find just my users that the permission is admin, and there is my query using CQRS, so my data was moved to my query stack,
27:02
and now, there is the admin permission with the description, and all the users that are an admin. Then the query, there is no joins. It's a flat query. So, now, I'm going to find my user, and there is. The other user, that's just a user. The usability is the same than
27:31
a normal application. So, to my user, it's not so different, and if you look at this, okay, it's the same that I have been using in my life, but the main point is now,
27:50
let's come back to this. If my normalised B fell down, it's broken for some motivation, or locked,
28:04
I don't stop at my query stack, and my users will consume the data, and if my query stack is over, I could write in stuff. Obviously, it's just a user service. It's not so critical,
28:28
but thinking due payment, you could receive a payment, but not retrieving the report about the payment, but you are selling the stuff yet. It's important, I think so. And
28:44
so, once, in another job that I have, the disk of the normalised DB broke, and we maintained the application running. It was like a monkey test. We maintained the application running
29:04
because all the writing process kept stopping on the queue, and was started in the queue. When we restarted the DB using the slave, we could reproduce all the stuff on the DB, and we don't lost the data, but it was just a moment that the normalised DB doesn't receive
29:24
the data, and it is amazing. It's very scalable, and maintainable, and, obviously, make your application stable. So, it's simple. Simple. It's not so hard.
29:43
So, common mistakes, and for common mistakes, there is a Jar Jar Binks! It's a common mistake! Jar Jar Binks is a common mistake. Okay, yeah, Jar Jar Lucas, thank you. So, but not Ross, the guy in the new movie. I don't know why Benicio del Toro did the
30:07
last Star Wars. Ross Jar Jar Binks. I don't know. Jar Jar Binks ate some insects, I don't know, some bugs. So, secure S and event sourcing must be applied together.
30:24
It's not true. It's a common mistake. There is an example that I'm just using CQRS. They have different proposals, and they work very well together, like Han Solo and Chewbacca,
30:40
but they have different proposals. One is to improve performance and scalability and stability, and another one is to create a streaming history about your data. So, event sourcing,
31:01
using event sourcing, you could reproduce the data in a specific moment of the time, and to know what happened. So, CQRS is going to improve your stability and performance, and make a better application for you. So, CQRS requires eventual consistency.
31:29
Eventual consistency is the most common usage. It's the most common scenario in CQRS, but if you try to use the first option of synchronisation that I've shown,
31:44
it's not eventual consistent. It's not totally eventual consistent. Try to be fast, but it's not mandatory, eventual consistency in CQRS. CQRS depends on queues and message brokers. It's a good practice, but not mandatory. So, you could use CQRS trying to
32:07
trigger at some events, or using logs from your data, like MySQL logs or something like this. There are many stages that you could use. I like to use queues and message brokers,
32:23
because I think that's easy to create an intervention or to monitor. CQRS is easy. It's not easy. And why? There are more things to monitor. There are 2DBs to take care.
32:44
There are many stuff in your stack that isn't in a normal application, the normal architecture that we looked at earlier. So, CQRS is not easy to implement,
33:07
but not so complex as we think. If good tools, you could have a good implementation. CQRS is architecture. No, it's not architecture. It's a pattern inside your architecture.
33:27
So, it's a good pattern that is cross-platform. It's not for Java, or just for Python, or for Go. It doesn't matter. It's for everyone to apply CQRS as a pattern. It's not an architecture.
33:49
My English teacher told me she's from Dublin and told me that's very funny. I don't know. CQRS is the best thing since sliced bread. And she told me, oh, it's so funny.
34:07
It's sliced bread! So, Sarah, thank you for the English class. I'm bad. And, again, there's my book. If you would like to learn something more than CQRS or other
34:24
patterns, there is an interesting pattern that I documented that the crashing first is interesting, and another pattern is internal patterns for service and communication patterns. It's very steeper than lunch here, and the dinner. So, they book obviously,
34:41
and there's a link. Please, give milk for my kids. I don't have kids, but, yeah, my cats. And may the CQRS be with you. And, okay? Thank you. Thank you. So, any questions? Please come up here. Okay. So, I have a question.
35:19
Okay. Oh, there is a question. Okay.
35:23
So, I have a question. I came in slightly late, so I probably missed it, which is my bad. I'm sorry. I maybe missed the very introduction, and also it's quite new to me. So, I'd like to clarify my understanding. So, what I glean from it is that you use two databases. You have your normalised database, which is your canonicalised data store, and you have an unnormalised database,
35:44
which is specifically optimised for the queries that you need. So, the straight queries can go to the de-normalised database, and that provides the primary advantage of the pattern you're advocating. Did I get that right? If I understand your question, yes. My idea is create the CQRS stack responsible just to
36:06
retrieve the data. So, it should be the most optimised as possible. And, you could apply cache and all the stuff over this stack. I don't apply to make the example, because I would like to make it simple to see, but you could take care of this CQRS stack,
36:27
this query DB, like all the other DB, but all the source of true DB. But, the main idea is optimise this segment of your stack as possible. And, does this mean the way you have
36:43
schema changes, you have to apply migrations to, or you have to apply migrations to both databases, where you have new queries, you have to generate a whole new set of data into your second database? Sure. My entities are different on both DBs. So, my entities, my
37:03
migrations, it's another stuff, okay? I'm doing this. Some people just try to reflect the same entities, but it's like presume flat queries, it's not mandatory to use flat
37:20
queries. If you try to use flat queries, it's, I see that you should, maybe you should create new entities specifically for the query stack. Yeah, it was a really interesting talk and I enjoyed it and learnt from it. Thank you. You're welcome. Okay, I'm
37:40
basically going for the same. So, when you are going to the right part that you did to scale and you said in the start of the talk, how are you actually going to scale? So, you're going to have different right instances, different right databases, with different schemas, or how is the approach for that? For which? For the command part, so. To
38:01
the command part, the main idea is command normally is thinking, I will use a practical example. MongoDB into version 2. To put something in MongoDB into version 2, block the read. Okay, into version 2, not more into version 3. And, but maybe for your
38:26
business, it's interesting to use the Mongo engine. So, using CQRS, you could totally divide this. The simple separation of stacks improves performance because now the writing
38:46
process wants to lock the query process. The scalability using master slaves and different instances and this kind of things is practically the same. The main goal of the
39:02
pattern is segregate the responsibilities to make this more resilient and improve performance to don't make that one operation lock the other. It's the main idea. But the instance control of the bees and this kind of stuff is the same approach. So, the right
39:22
improvement. So, how you're actually going to scale the right part. That's what I mean. So, you're going to queue and you want to deal with a queue that if it's delayed, the queue will be taking or? There, there, you could apply chargings as an example that will be very optimized and is not so optimized in some engines for CQRS stack because what
39:45
happened with some engines using charging for DB, you have an instance that's responsible to administrate all the, the other DBs that comes, is going to write because that's more idle. We'll get the request and write on DB. But to find maybe is not so
40:05
interesting because you're going to receive the request by the same charging and that should find in various the data why all this data was not propagated for all the chargings. And in using CQRS, it's not an issue. And by because you are not interesting to use this
40:28
collected data from the charging because the charging is just a source of true. It's, it's improved performance on the writing segment. And your data is a stream it on the Kafka or web MQ or active MQ, some other. And it was right in the CQRS stack that
40:44
responsible to retrieve the data. So, that could have another approach to scalability different than charging. And it's just an example about, oh, you could use charging, you could use master slave decide, you could use all the stuff that is fast to read, but is slowly to get data. And this kind of approach is not more, more an issue if your business requires too
41:06
much retrieve, too much search. Okay. Okay. Thank you. Hi. In the, when you were talking about the different strategies to synchronize the two stacks, you talk about the
41:25
explain and then you talk, you talk about the, that the, if I understand good that the query stack ask for an update when he knows that it's outdated, it's how he knows that it's outdated.
41:43
There are some approach to use this. It's the more harder to apply. It's the hard way to apply. And it's very, usually the people create like a semaphore in the streaming too, like on the streaming, there is this, you put a semaphore on the Kafka and okay,
42:04
I'm very outdated from this because the data stream is, it's there or it's full and I'm going to get the information from the common stack. And there are the logs of the DB.
42:21
That you could consume in this and the check, okay, my DB logs are very full. I don't, I'm moving DB logs from different segments of your, your server. And okay, there is, it's full of logs and they should consume these logs to create my, to improve my query
42:42
stack. There are two different approach. One is that's more common. Okay. Once it's looking the, the bus and to see if there is a sign out to consume the data. And another one is looking the DB logs that it was, you reserve this specific part and a very common approach is
43:07
using like a JVM separation. That's the Eden and the old part of DB logs. And when you consume the data, you move the Eden to the old part of the logs. And if the Eden is full,
43:20
it's time to get the data again. And the logs is optimized to use to element, to bring the sufficient information to register on the query stack. Was clear or was, yeah, we could talk, it's hard, we could talk and more common, you'll be interesting. More questions? Okay.
43:45
Thanks for the talk. I really enjoyed it. It seems to me that when writing to say the local flat database or the Mongo DB, whatever it is, your data predominantly comes from the events, at least in this, in this example. However, when you start a new application running,
44:04
a new service, presumably or in other scenarios, you need to be able to read the data from the relational database that stores the conical representation. And it seems to me that there's some complexity there in the sense that you need to have, you need to be able to write to
44:22
local database using two different sources of information. You need to be able to have the logic to do the writes based on events or logic to do the writes based on the relational database. I was wondering if you find that's a problem for you or how you manage that complexity or those two different kind of pieces of logic in your implementations.
44:42
Yes, it's one of the drawbacks using CQRS. Mainly if you want to create specialized queries, if you just want to replicate and segregate the search and the find and the write process, it's okay. It's not a consequence. But if you want to create flat queries,
45:08
it will be a consequence. You will have the two different, I'm going to show you something here. If you look at my model, specifically for this business, I'm using a user. And
45:29
there is the, it's my postgres, okay, two tables. And there is a document for this user that's going to get the user information, just user. And I create a struct to create
45:46
an embedded document here. So I have different entities and I have different logics. And it will be a normal scenario inside CQRS if you're trying to create flat queries. The mistake that you should take care of is don't split the domain rules between the
46:09
event component and the domain. So it's a normal mistake. Here on my domain, I prepared the
46:24
data to be consumed by my event component. Okay? All the data. I'm dispatching two events. That's not a good practice. But I'm dispatching just to show this stuff. But
46:42
it's very common to see something like this. Oh, I would like to create a relationship between permissions and users. And I don't want to have both entities being consumed in the same domain. So it's like consume two repositories in the same domain. And the people
47:01
move the second part to here. And it's a real mistake because you start to split the domain. And it's a mistake. But two different entities, two different logics, one specialized to write and another one specialized to read is exactly what CQRS means. Okay?
47:29
Thank you again. And so enjoy your lunch. Thank you. I'm able to answer you. Thank you.
Recommendations
Series of 5 media