​Recurring Background Jobs with Sidekiq-scheduler

Video in TIB AV-Portal: ​Recurring Background Jobs with Sidekiq-scheduler

Formal Metadata

​Recurring Background Jobs with Sidekiq-scheduler
Title of Series
Part Number
Number of Parts
CC Attribution - ShareAlike 3.0 Unported:
You are free to use, adapt and copy, distribute and transmit the work or content in adapted or unchanged form for any legal and non-commercial purpose as long as the work is attributed to the author in the manner specified by the author or licensor and the work or content is shared also in adapted form only under the conditions of this license.
Release Date

Content Metadata

Subject Area
When background job processing needs arise, Sidekiq is the de facto choice. It's a great tool which has been around for years, but it doesn't provide recurring job processing out of the box. sidekiq-scheduler fills that gap, it's a Sidekiq extension to run background jobs in a recurring manner. 
In this talk, we'll cover how sidekiq-scheduler does its job, different use cases, challenges when running on distributed environments, its future, how we distribute capacity over open source initiatives, and as a bonus, how to write your own Sidekiq extensions.
Computer animation Scalar field Extension (kinesiology)
Scalar field Office suite
Computer animation Personal digital assistant Operator (mathematics) Energy level Planning Quicksort Traffic reporting Task (computing)
Revision control Logical constant Process (computing) Computer animation Multiplication sign American Physical Society Energy level Water vapor Maxima and minima Task (computing) Physical system
Scaling (geometry) Computer animation Computer configuration Cellular automaton Multiplication sign Scalar field Universe (mathematics) Endliche Modelltheorie Whiteboard Service-oriented architecture Local ring
Type theory Dependent and independent variables Digital photography Computer animation Open source Whiteboard Graph coloring
Point (geometry) Scaling (geometry) Computer animation Blog Multiplication sign Active contour model Demoscene
Cellular automaton View (database) Set (mathematics) Planning Neuroinformatik Type theory Computer animation Personal digital assistant Scalar field Speech synthesis Energy level Cycle (graph theory) Office suite
Type theory Multiplication Computer animation Chemical equation ACID Bit Instance (computer science) Endliche Modelltheorie Quicksort Resultant
Computer animation Multiplication sign Videoconferencing Website Instance (computer science) Collision Quicksort Mereology Bounded variation
Dataflow Context awareness Roundness (object) Multiplication sign View (database) Moment (mathematics) Bit Office suite Theory
Doppler-Effekt Process (computing)
Scheduling (computing) Server (computing) Group action Arm Physical law Event horizon Formal language Process (computing) Computer animation Hooking Personal digital assistant Computer configuration Electronic visual display output Video game Cycle (graph theory) Extension (kinesiology)
Collaborationism Scheduling (computing) Computer animation Open source Multiplication sign Feedback Projective plane Online help
Android (robot) Context awareness Scheduling (computing) Hoax Open source State of matter Multiplication sign Ultraviolet photoelectron spectroscopy Set (mathematics) Rule of inference Programmschleife Operating system Software framework Software testing Error message Conditional probability Exception handling Arm Electric generator Scaling (geometry) Inheritance (object-oriented programming) File format Representational state transfer Performance appraisal Loop (music) Computer animation Personal digital assistant Library (computing)
the mind so a minor Austin well beyond so this is a sponsor talk but move it had a company worker and a lot of flak about like scalar I want to psychic extension for the an extension for psychic had
no ego so my name is Andreas and this is John I where you we work for movies that were based on South America and your wife and I yeah we've been growing we've got people Argentina Colombia and we also about an office in Austin if you're in Austin we're we're also always hiring so have which is not and yes so
psychic scalar that's that's where it's hosted and I'll leave you with John now for a while the I explain a little more about what it is it so let's start with you allocable
use cases that it will come in 2005
we were working on a person that among other things near to the that's important to the great level scores basal financial data on a plane that is to by candidates also operators nice we need to send we adapted to summon his reports bookable of the 1st approach which we sort of what was to set up currently is which will execute the trade tasks in which if you'd pay does
perform the processes the that emission for with the water is an approach but with some works so when it's gonna
the gasses the possible that whistles is for most of the most needs to start to interpret that In each run we where we are using J 2 we and take some time to start up on consumes advantage of Melanie so if we if we call currently up size crime these at so what's the BN for somebody is willing to start on Jaitly process and constant from a money that's always the way to programmatically up the move and they would understand gone entities goes cooperation we need to live outside of the APS cooperation yeah it's another Bordeaux solution because interlaced on the British system and you can start on stub the skill level without affecting other current there are also some of it's like Gross minimal solution is that minute 11 not that's a conflict and underplaying the up in a blaster we don't appear the be David tasks when in fact you need to run on once this but those those
locks we did discover from but we saw that there was some of the solutions we followed him into cells and so on nanosecond whose purpose was to a scalar thank excels at that specific time in the future the this wasn't what we were looking for but we still ourselves how and because some kind support to find its scope the the setting options we stumble brokers of Canada which is session where you can use constant that's the scale a quality of local fruit sold them up with any of indigenous was into and into thinking scalar are in that way across the board that was in Latinos 2 cells and so on and after doing that we start policing forgive scalar University we were working on into cells obtained models for the crop to to up to maintain the gem FIL the the federal owner to us I don't mean to Sosa 16 without support for themselves when forget he said the US adduces began so how do you use
think is given as things go on is
just Anastasia like applying to cite to fight you it is not the belief and then is a psychic but the fall we need to declare of a psyching work which is ugly glass including psychic worker emoluments and that was a response to that perform useful the this can competition is places inside confirmation fight and this can key so in that example with color the lower that device every minute when say even to see all and when the that it the city planning Damocles psychic Skoda we push the pedal warm Joe into psychic it OK then you fairly general cars genocide or if what you want to use by the just Iet intuition but usual the then a and we psychic askew Photo source of we have hindered some we are outside the phrase we have a glaring we have psychic to require our allude to find that contains toward a plus the with on that sits in a minute a Sec is going to in Culatello on this idea we perform the show the it very
different this can direct us honest we rely on rubles which is 710 10 it a civilian the different going types of the board they're going
online the most common 1 most of the other 1 is
is gone a which that's enemies on kind of a and it's at the same ground like syntax that yet he said to a contour leading into the stumbling to and you and you sitting inside the scale it out died which proposes a was that the Pacific point in time every that's it it was assaulting according wide way put it in Newfoundland scene and in died which pushes the show once up the sometimes relational has elapsed the the of the main blog also
cytoskeleton is to push cells into psyche this is then the main idea it is psychic then on those shows us usually focal set singing the scalar works the Wednesday
and Wednesday did a scalable is quite when you requires is it cooks into a start-up and sit down say let's let's take me for we will use this plane then it will those led cycle even so all faces are then under a the on this type of face no we that's the population billable creation and is that starts before scalar which is before scary so just after that candidates all we're all in this case was found invoke speech really blow when 1 is a with that was carried out and said that's as kilojoule for every 1 of the computer 8 of them views of is high level in the offices is of a positive for start executing for into layers we find that in case of ground and add salt in that so if that was not previously bullets of finally push the showing to psychic leading psychic thundershowers useful the
and when implement this approach we stumble upon the challenge of managing with the balanced model instance with a bit than instances and the sentence at 1st but the results were meant to on one-sided
concerns only 1 unaccommodable those notes were needed only 1 well-known rather think it is given when they are we host various acidic instances Danny shows was Mason in 1 support for multiple nodes was sort later foreground and at types so right now it's possible to run with the defective alleles a scalar instances having grown on that so those Cronin as well so we know that the data all right well there the otherwise I'll just out of the sorry the 1 that I mentioned earlier about the future of which yeah honestly but the 1 thing that we want
to do is we want to start the people that the duration of right now we have a lot of variation in into the site of the and that you know it from society itself we tried to surmise that we possibly have collision there so we wanna the way I don't wanna learn again every and instead last part to work with multiple instances of you so you can you don't have to worry about that and well so so 1 of the things we don't want to do is we don't want to work on-site at the stage where we start it is outside the scope of just scaling right so 1 of the things it you just you know that out of nothing more time so also there's some that are going to be done and all of this it right what time now I
it may a video of this sort of you know
so that works can hopefully so I really I quite yet we had in mind were in 1 time of war aware of the importance of the if you have the value it and if you have them you know make use and let's say that you know you the last moment and it was from you and you're not really find that that if you and I may be a false inactivity and theories of flows there may be a lot of that really wasn't there about anymore but we but it maybe after a month or 2 you might wanna automatically loaded and so we we we thought was the general idea we have offices in the month is that the left the that was the only thing is you have the invalid and the within mark the huge so we created our a you know a little bit loading by it will be there runs every minute and text those those Margaret Hughes and the of them so far with right now you can do that
for of a that will small have less likely the dealer and the the we have a lot of theory in view of the of the of the you run every minute um and but in a minute but if you think of the of of funding will find you because all them and so that you also believe me when I go over to the round out the the they were
well the obviously if I if nothing
else is open at the job or on and they will do nothing so will not find any issue and so if we go back and now let's say a few months fast
and uh no 1 no activity was on that issue uh you know it will will simulate that marketed as a Doppler catering and and so the next the job
runcible just clean it up OK I so as a bonus to start
we want to show you have what it takes or what tools you have available to the rider a psychic extension on so you get these 4
events of in the that psychic life cycle that you can have a look into I like John mentioned earlier we hope into start-up and shutdown events so that's where we so a and we build up the configuration and start doing the of the scheduler arm and then in a in a shutdown of the the law generalized you know it was not clean up the thread and and I and then I 1 thing that I think we didn't display here but when you run out the weather extension of psychic scheduler will actually have and create an not attack for you and it will show you the the the jobs that are currently enqueued and it will also let you disable them so you you have you know without logging into the server you can control it via the web and and so to do that you you just gotta create like a module extension a where you gonna hook into that that method called self that registered and and then you can do with that Apple get inputs by region the put the path and there in itself it's a way for you to you know define actions and return is similar or what whatever data you wanna rendered here on on that have and once you do that you have to initialize the I to research the extension on psychic so psyche where you get a registered and you can add a tab and you put the name and the path that that it points to and the path really is is much but here when you do not don't get my X that's that's a path you get user and yeah and you also have the option of adding lokale so in case supported various languages I was psychics scheduler it's just those for I think a more OK so she's just showing for and so you do I know for warning 1 language on and I yeah so this this this example is is up on the on move it um not your l although if you want to gather side extension may wanna check outside its schedule um alright
now about feedback and collaboration and as is this is obviously an open source project we it
has lot since I mean weighted created we just you know took over amendments of a few years ago or so over the course of the years it has had 43 contributors and in right now we have like 6 hours a week allocated for issues so if you find anything wrong with that With that psychic scheduler you'll get some time for us on to you know to fix it and I was always help and and feedback is very appreciated that it and as a last thing of
we have some other open source and libraries are we get our recent which is as simple as exception notification genome that that you can use for a lot on sending errors are in any will be out of that you have all that we have really which is a lightweight rules evaluator for context Irving conditional expressions and will suffer the arm then we have Angus on it's so that the name for that was was a real story and uh in Uruguay whereas we're loving country we like to eat a lot of me and saying is you know obviously is a famous on rule 1 time and but it really does not to do any anything with that so we just thought it was cool out what it is it's a arrest like API framework and it it will it will run rack so if you you already using a arrived at you can use that to to serve your API and and it already generates a so while you're writing it kind of generates documentation on the way of and I'll yet so you can use that for RESTful API in and then we have figured which is not really something for Ruby it's up is an android and fate data fake but realistic data generators so you can generate names emails states and a country names which you know you and will be take but it will become a real so on you know you don't wanna generators not just a few letters or e-mail you need you need a certain format yeah so but that is and test basically parents at the
for us the short few but the question is how do we manage the the 2nd resolution but within this scheduler well practice and we use proposed scale for that but was just the start ups of editing and the set some gives semicircles be alive hello folks think it's checks it's loops through all the skidoos aren't air ask is this can this scale has to run and in that case it constantly block so it's it's not just like Hooke's with somebody and from there it from the operative system sets make some who is the it's loop then is lead to something like that in so it's that that in it that means that your will be built we not to run out it should be and since account said on the sequence but that it will break up kind then 19 this is not it's not the best time the left yeah yeah the what would assure the each with writing that secondly that sick the all right thank you very much have that so Active
things so in his his