openstackgerrit | Ian Wienand proposed openstack-infra/zuul master: nodepool-integration fixes https://review.openstack.org/545163 | 00:05 |
---|---|---|
openstackgerrit | Ian Wienand proposed openstack-infra/zuul master: nodepool-integration fixes https://review.openstack.org/545163 | 00:22 |
SpamapS | I feel like there's something that I've done wrong that has made my nodepool and zuul relationship very laggy | 00:24 |
SpamapS | I have plenty of ready nodes sometimes... but it takes 5+ minutes for that request to be fulfilled | 00:24 |
clarkb | are the ready nodes all in one provider and the other provider is grabbing the request locks first? | 00:25 |
SpamapS | yes | 00:26 |
SpamapS | oh wait no | 00:26 |
SpamapS | I misread that | 00:26 |
SpamapS | I have two providers, one set at 0 max-servers, the other at 75 | 00:26 |
clarkb | the other thing to watch out for is ready nodes label not matching the request node types so they are ignored | 00:26 |
SpamapS | (this is so that I can fail over if need be) | 00:26 |
SpamapS | So most requests say they were rejected by the 0-'d provider fairly quickly | 00:27 |
SpamapS | No in the logs I see the launcher saying it has them as Active | 00:27 |
SpamapS | just seems like they don't get into a state that they're given back to Zuul for a long time | 00:27 |
SpamapS | maybe there's stuff not in the log that it's doing that is important. | 00:27 |
clarkb | what should happen roughly is both provider threads in nodepool poll for new requests. First one with max-servers 0 will decline it due to max servers count. The second will then pick it up and if it has enough ready nodes of the correct type it will hand those over to zuul immediately. Otherwise it boots new instances and then hands them over | 00:29 |
clarkb | if booting new instances ~5 minutes is probably not an uncommon time to ready period | 00:29 |
clarkb | but I would expect no more than a few seconds if handing over existing nodes | 00:29 |
clarkb | SpamapS: I would check and see if the ready nodes are actually being used or if they are "stuck" and getting stale (probably the timestamp associated with them is sufficient to tell) | 00:31 |
SpamapS | clarkb: often I see nodes built for the request, and then they sit in 'ready' for a few minutes | 00:35 |
SpamapS | I have a 5-node job, and 3 AZ's, so even with 15 min-ready, sometimes I don't have 5 in the zone it chooses, and it has to build | 00:35 |
SpamapS | instances usually boot within 2 minutes.. but yeah, just seems like it sits there for a while after that | 00:36 |
SpamapS | what I really need is to setup a statsd and see what those stats tell me | 00:36 |
openstackgerrit | Ian Wienand proposed openstack-infra/zuul master: nodepool-integration fixes https://review.openstack.org/545163 | 00:38 |
openstackgerrit | Paul Belanger proposed openstack-infra/nodepool master: Use $NODEPOOL_KEY_NAME for dsvm testing https://review.openstack.org/545176 | 00:39 |
pabelanger | SpamapS: yah, statsd should help see what is going on. | 00:40 |
clarkb | also the ready -> in-use transition is done by zuul when it actualyl starts to execute the job | 00:42 |
clarkb | could be some slowness on the zuul side maybe? | 00:42 |
clarkb | "Executing jobs for change %s" is the debug string for when that happens | 00:43 |
clarkb | maybe see hwo far apart those are | 00:43 |
*** elyezer has quit IRC | 00:46 | |
*** elyezer has joined #zuul | 00:46 | |
openstackgerrit | Ian Wienand proposed openstack-infra/zuul master: nodepool-integration fixes https://review.openstack.org/545163 | 00:49 |
*** persia has joined #zuul | 00:54 | |
*** elyezer has quit IRC | 00:56 | |
*** elyezer has joined #zuul | 00:57 | |
openstackgerrit | James E. Blair proposed openstack-infra/zuul master: Make NodeSetParser a regular class https://review.openstack.org/545148 | 01:00 |
openstackgerrit | James E. Blair proposed openstack-infra/zuul master: Make SecretParser a regular class https://review.openstack.org/545149 | 01:00 |
openstackgerrit | James E. Blair proposed openstack-infra/zuul master: Make JobParser a regular class https://review.openstack.org/545150 | 01:00 |
openstackgerrit | James E. Blair proposed openstack-infra/zuul master: Make PipelineParser a regular class https://review.openstack.org/545151 | 01:00 |
openstackgerrit | James E. Blair proposed openstack-infra/zuul master: Make SemaphoreParser a regular class https://review.openstack.org/545152 | 01:00 |
openstackgerrit | James E. Blair proposed openstack-infra/zuul master: Make TenantParser a regular class https://review.openstack.org/545153 | 01:00 |
openstackgerrit | James E. Blair proposed openstack-infra/zuul master: Add ConfigLoader initializer https://review.openstack.org/545180 | 01:00 |
openstackgerrit | James E. Blair proposed openstack-infra/zuul master: Remove tenant argument from UnparsedTenantConfig https://review.openstack.org/545181 | 01:00 |
*** rlandy|rover|bbl is now known as rlandy|rover | 01:07 | |
openstackgerrit | Ian Wienand proposed openstack-infra/nodepool master: Add legacy-copy-jenkins-scripts https://review.openstack.org/545158 | 01:12 |
openstackgerrit | Ian Wienand proposed openstack-infra/nodepool master: Add legacy-copy-jenkins-scripts https://review.openstack.org/545158 | 01:14 |
openstackgerrit | Paul Belanger proposed openstack-infra/nodepool master: Use $NODEPOOL_KEY_NAME for dsvm testing https://review.openstack.org/545176 | 01:18 |
openstackgerrit | Paul Belanger proposed openstack-infra/nodepool master: Add opensuse-tumbleweed to nodepool dsvm testing https://review.openstack.org/545183 | 01:18 |
*** JasonCL has quit IRC | 01:19 | |
*** JasonCL has joined #zuul | 01:20 | |
openstackgerrit | Ian Wienand proposed openstack-infra/nodepool master: Add legacy-copy-jenkins-scripts https://review.openstack.org/545158 | 01:27 |
*** rlandy|rover is now known as rlandy|rover|bbl | 01:31 | |
*** JasonCL has quit IRC | 01:36 | |
*** JasonCL has joined #zuul | 01:37 | |
*** JasonCL has quit IRC | 01:41 | |
openstackgerrit | Ian Wienand proposed openstack-infra/nodepool master: Add legacy-copy-jenkins-scripts https://review.openstack.org/545158 | 01:45 |
*** elyezer has quit IRC | 01:55 | |
*** elyezer has joined #zuul | 01:57 | |
*** elyezer has quit IRC | 02:05 | |
*** elyezer has joined #zuul | 02:06 | |
openstackgerrit | Ian Wienand proposed openstack-infra/nodepool master: Add legacy-copy-jenkins-scripts https://review.openstack.org/545158 | 02:23 |
openstackgerrit | Ian Wienand proposed openstack-infra/nodepool master: Add legacy-copy-jenkins-scripts https://review.openstack.org/545158 | 02:38 |
openstackgerrit | Ian Wienand proposed openstack-infra/nodepool master: Add legacy-copy-jenkins-scripts https://review.openstack.org/545158 | 02:46 |
openstackgerrit | Ian Wienand proposed openstack-infra/nodepool master: Add legacy-copy-jenkins-scripts https://review.openstack.org/545158 | 02:59 |
openstackgerrit | Ian Wienand proposed openstack-infra/nodepool master: Add legacy-copy-jenkins-scripts https://review.openstack.org/545158 | 03:18 |
*** harlowja_ has quit IRC | 03:23 | |
*** rlandy|rover|bbl is now known as rlandy|rover | 03:27 | |
openstackgerrit | Ian Wienand proposed openstack-infra/zuul master: nodepool-integration fixes https://review.openstack.org/545163 | 03:28 |
*** rlandy|rover has quit IRC | 03:29 | |
*** elyezer has quit IRC | 03:34 | |
*** elyezer has joined #zuul | 03:37 | |
*** jaianshu has joined #zuul | 03:46 | |
*** JasonCL has joined #zuul | 03:55 | |
*** JasonCL has quit IRC | 04:00 | |
openstackgerrit | Ian Wienand proposed openstack-infra/zuul master: Fix nodepool-zuul-functional https://review.openstack.org/545163 | 04:33 |
openstackgerrit | Ian Wienand proposed openstack-infra/nodepool master: Fix nodepool-zuul-functional https://review.openstack.org/545158 | 04:33 |
*** harlowja has joined #zuul | 04:46 | |
openstackgerrit | Ian Wienand proposed openstack-infra/zuul master: Fix nodepool-zuul-functional https://review.openstack.org/545163 | 04:50 |
openstackgerrit | Ian Wienand proposed openstack-infra/zuul master: Add nodepool-zuul-functional as non-voting check job https://review.openstack.org/545203 | 04:50 |
*** harlowja has quit IRC | 04:59 | |
*** harlowja has joined #zuul | 05:19 | |
*** harlowja has quit IRC | 05:27 | |
openstackgerrit | Merged openstack-infra/nodepool master: Use $NODEPOOL_KEY_NAME for dsvm testing https://review.openstack.org/545176 | 05:44 |
*** abadger1999 has quit IRC | 06:13 | |
*** abadger1999 has joined #zuul | 06:13 | |
*** threestrands has quit IRC | 06:20 | |
*** openstackstatus has quit IRC | 06:27 | |
*** openstack has joined #zuul | 06:31 | |
*** ChanServ sets mode: +o openstack | 06:31 | |
*** elyezer has quit IRC | 06:53 | |
*** elyezer has joined #zuul | 06:57 | |
openstackgerrit | Andreas Jaeger proposed openstack-infra/nodepool master: Use bindep for nodepool-zuul-functional https://review.openstack.org/545219 | 07:06 |
*** elyezer has quit IRC | 07:07 | |
openstackgerrit | Andreas Jaeger proposed openstack-infra/zuul master: Remove install-distro step for nodepool integration https://review.openstack.org/545221 | 07:08 |
*** elyezer has joined #zuul | 07:08 | |
openstackgerrit | Andreas Jaeger proposed openstack-infra/nodepool master: Remove copying of legacy install-distro script https://review.openstack.org/545222 | 07:09 |
*** threestrands has joined #zuul | 07:22 | |
*** threestrands has quit IRC | 07:22 | |
*** threestrands has joined #zuul | 07:23 | |
*** threestrands has quit IRC | 07:23 | |
*** threestrands has joined #zuul | 07:23 | |
*** JasonCL has joined #zuul | 07:27 | |
*** JasonCL has quit IRC | 07:32 | |
*** threestrands has quit IRC | 07:34 | |
openstackgerrit | Andreas Jaeger proposed openstack-infra/nodepool master: Fix nodepool-zuul-functional https://review.openstack.org/545158 | 07:38 |
openstackgerrit | Andreas Jaeger proposed openstack-infra/nodepool master: Fix nodepool-zuul-functional https://review.openstack.org/545158 | 07:39 |
openstackgerrit | Andreas Jaeger proposed openstack-infra/zuul master: Remove install-distro step for nodepool integration https://review.openstack.org/545221 | 07:40 |
openstackgerrit | Andreas Jaeger proposed openstack-infra/nodepool master: Remove copying of legacy install-distro script https://review.openstack.org/545222 | 07:40 |
AJaeger | ianw: but yes, once we remove zuul-cloner for non-legacy jobs, it will fail again - so needs to be done | 07:41 |
AJaeger | ianw: let's merge tools/nodepool-integration-setup.sh into the ansible playbook | 07:44 |
openstackgerrit | Andreas Jaeger proposed openstack-infra/nodepool master: Avoid tools/nodepool-integration-setup.sh from zuul https://review.openstack.org/545228 | 07:52 |
openstackgerrit | Andreas Jaeger proposed openstack-infra/zuul master: Remove tools/nodepool-integration-setup.sh https://review.openstack.org/545229 | 07:52 |
*** elyezer has quit IRC | 07:55 | |
*** elyezer has joined #zuul | 07:56 | |
openstackgerrit | Andreas Jaeger proposed openstack-infra/nodepool master: Avoid tools/nodepool-integration-setup.sh from zuul https://review.openstack.org/545228 | 07:57 |
openstackgerrit | Andreas Jaeger proposed openstack-infra/zuul master: Remove install-distro step for nodepool integration https://review.openstack.org/545221 | 07:58 |
openstackgerrit | Andreas Jaeger proposed openstack-infra/zuul master: Remove tools/nodepool-integration-setup.sh https://review.openstack.org/545229 | 07:58 |
*** hashar has joined #zuul | 07:58 | |
openstackgerrit | Andrea Frittoli proposed openstack-infra/zuul-jobs master: Improve file renaming in stage-output https://review.openstack.org/544606 | 08:04 |
*** jpena|off is now known as jpena | 08:41 | |
*** JasonCL has joined #zuul | 09:32 | |
*** JasonCL has quit IRC | 09:38 | |
*** openstackgerrit has quit IRC | 10:18 | |
*** JasonCL has joined #zuul | 10:26 | |
*** elyezer has quit IRC | 10:27 | |
*** JasonCL has quit IRC | 10:31 | |
*** elyezer has joined #zuul | 10:40 | |
*** JasonCL has joined #zuul | 10:52 | |
*** JasonCL has quit IRC | 10:57 | |
*** JasonCL has joined #zuul | 11:04 | |
*** elyezer has quit IRC | 11:33 | |
*** elyezer has joined #zuul | 11:41 | |
*** jaianshu has quit IRC | 11:49 | |
*** elyezer has quit IRC | 11:53 | |
*** elyezer has joined #zuul | 11:55 | |
kklimonda | I have a small problem with zuul-scheduler - executor running jobs died (a problem with logrotate) and never reported back job results, and now scheduler just waits for them - even `lookForLostBuilds` is not doing anything - I'm running a "bit" behind master so it's quite possible that this has been fixed, but is there anything I can do to report to scheduler that those jobs are gone, or do should I just restart it? | 12:28 |
kklimonda | I can probably upload a new patchset, and that should make scheduler cancel current jobs | 12:39 |
rcarrillocruz | someone mentioned ( i think pabelanger ? ) that there was a patch for dequeue | 12:46 |
rcarrillocruz | not sure if that would help, but still, a patch, nothing in tree | 12:46 |
rcarrillocruz | zuul client just have enqueue/promote | 12:46 |
*** elyezer has quit IRC | 12:48 | |
*** jpena is now known as jpena|lunch | 12:48 | |
*** elyezer has joined #zuul | 12:51 | |
*** openstackgerrit has joined #zuul | 12:58 | |
openstackgerrit | Ricardo Carrillo Cruz proposed openstack-infra/nodepool master: WIP Add Nodepool Ansible generic driver https://review.openstack.org/545286 | 12:58 |
* Shrews waves hello and attempts to catch up on all-the-things | 13:04 | |
*** rlandy has joined #zuul | 13:37 | |
*** rlandy is now known as rlandy|rover | 13:37 | |
*** jpena|lunch is now known as jpena | 13:51 | |
pabelanger | kklimonda: yah, that is a known issue. Trying to find storey ATM. But, need to restart scheduler currently to clear it. | 14:01 |
pabelanger | kklimonda: i think it is https://storyboard.openstack.org/#!/story/2000899 | 14:04 |
kklimonda | pabelanger: thanks, I'll try to get someone to upload a new patchset and see if scheduler decides to restart jobs, but will probably have to find a quiet moment to restart scheduler itself. | 14:18 |
kklimonda | I'm not quite sure what exactly is zuul doing when "Looking for lost builds" messages is printed | 14:18 |
AJaeger | ianw yesterday started fixing the nodepool-zuul-integration tests and I piled on top of that - reviews of https://review.openstack.org/#/c/545163/ and https://review.openstack.org/#/c/545158/ are welcome by nodepool and zuul cores, please | 15:38 |
Shrews | AJaeger: i have to go give someone a ride, but will review when i return. also, one of ianw's fixes duplicates my https://review.openstack.org/540983 which already has two +2s | 15:47 |
Shrews | not sure why it wasn't approved | 15:47 |
Shrews | biab | 15:47 |
AJaeger | Shrews: approve it yourself once you're back... | 15:47 |
AJaeger | I guess it was not approved since it still fails | 15:47 |
AJaeger | I can rebase on yours... | 15:48 |
pabelanger | Hmm | 15:51 |
pabelanger | anybody else get randon disconnects doing | 15:51 |
pabelanger | finger 5cd0434d23244c36a24532f1b102891b@zuul.openstack.org | 15:51 |
pabelanger | randomly I get stream internal error | 15:51 |
openstackgerrit | Andreas Jaeger proposed openstack-infra/zuul master: Fix nodepool-zuul-functional https://review.openstack.org/545163 | 15:52 |
openstackgerrit | Andreas Jaeger proposed openstack-infra/zuul master: Add nodepool-zuul-functional as non-voting check job https://review.openstack.org/545203 | 15:52 |
pabelanger | sorry, Internal stream error | 15:53 |
pabelanger | then connection drops | 15:54 |
openstackgerrit | Andreas Jaeger proposed openstack-infra/zuul master: Fix nodepool integration tests https://review.openstack.org/540983 | 15:55 |
openstackgerrit | Andreas Jaeger proposed openstack-infra/zuul master: Fix nodepool-zuul-functional https://review.openstack.org/545163 | 15:56 |
openstackgerrit | Andreas Jaeger proposed openstack-infra/zuul master: Remove install-distro step for nodepool integration https://review.openstack.org/545221 | 15:59 |
openstackgerrit | Andreas Jaeger proposed openstack-infra/zuul master: Remove tools/nodepool-integration-setup.sh https://review.openstack.org/545229 | 15:59 |
openstackgerrit | Matthieu Huin proposed openstack-infra/zuul master: zuul web: add admin endpoint, enqueue & autohold commands https://review.openstack.org/539004 | 16:11 |
*** elyezer has quit IRC | 16:11 | |
*** elyezer has joined #zuul | 16:14 | |
*** elyezer has quit IRC | 17:04 | |
*** elyezer has joined #zuul | 17:16 | |
clarkb | thinking about post-run timeouts. One super simple fix would be to make post-run's timeout max(30 minutes, remaineder of job timeout) and just say you have half an hour to get that done | 17:36 |
clarkb | not a super fan of hardcoding here but thats like a failsafe for that case where you had your normal timeout run out | 17:36 |
pabelanger | we could also create pre-timeout, timeout, post-timeout settings for jobs, with sane defaults | 17:37 |
clarkb | one concern I have with ^ is that we use pre like run in some places. iirc shade jobs run devstack in pre | 17:38 |
clarkb | I kind of see pre and run being in the same timeout class as a result | 17:38 |
pabelanger | http://paste.openstack.org/show/674728/ | 17:40 |
pabelanger | explains Internal stream error | 17:40 |
pabelanger | nodepool jobs have sleep 10 | 17:41 |
pabelanger | and nothing is transmitted across stream | 17:41 |
clarkb | pabelanger: so the jobs not writing anything results in the stream having no data to write? | 17:42 |
*** harlowja has joined #zuul | 17:55 | |
pabelanger | clarkb: yah, think so | 17:55 |
pabelanger | I can look more in a moment | 17:55 |
Shrews | pabelanger: i can see that happening if an executor finger daemon is gone due to oom | 17:57 |
Shrews | but maybe that would happen sooner in the process... on connect. hrm | 17:58 |
pabelanger | Shrews: we actually shouldn't have the OOm issues any more, we replaced kernels this week. And so far no OOM | 17:58 |
Shrews | orly? | 17:59 |
pabelanger | but in this case, stream process is still running | 17:59 |
pabelanger | Shrews: https://review.openstack.org/544164/ | 17:59 |
Shrews | well w00t w00t | 18:00 |
*** jpena is now known as jpena|off | 18:09 | |
openstackgerrit | Merged openstack-infra/zuul master: Fix nodepool integration tests https://review.openstack.org/540983 | 18:12 |
corvus | clarkb, pabelanger: so maybe we should just add a post-timeout attribute? | 18:19 |
pabelanger | I'd be okay with that | 18:20 |
AJaeger | corvus: yes - perhaps with a smaller limit than our current timeout? | 18:21 |
corvus | i don't think we have to have immediate symmetry. pre and post really serve different functions, and pre+run certainly do have a tight relationship. if we added a pre-timeout, i'd suggest maybe have it default to None, and that means the current behavior of "share the run timeout". but then if you specify a pre-timeout, it would only apply to pre (that way you could say "if it takes more than 10m to | 18:21 |
corvus | download from pypi, abort") | 18:21 |
AJaeger | especially on a 3 hour job ;) | 18:22 |
AJaeger | I'm fine with post-timeout for now | 18:23 |
corvus | and post-timeout, in all cases, only covers post. that way, we set on the base job a 30m default post timeout. it should never take longer than 30m to upload logs or publish a tarball or whatever. | 18:23 |
corvus | but if there's a 'publish diskimage to international space station' job, maybe we up it's post-timeout. | 18:23 |
clarkb | ya that makes sense to me | 18:24 |
corvus | clarkb: do you want to hack on that? | 18:25 |
clarkb | corvus: ya I can take a stab at it | 18:25 |
corvus | cool, thanks | 18:26 |
pabelanger | 789+56666666666666666666666666666666666666-************************************************************+6333333333333333333333333333 | 18:27 |
corvus | pabelanger: you okay? | 18:28 |
AJaeger | one * is multiplicate, ** is exponentiation but what's that crazy number of *s? ;) | 18:29 |
mordred | corvus: reading that - I kind of wonder if maybe post-timeout is a timeout for *each* post playbook would match the model better. if we timeout uploading a tarball we still want to upload logs | 18:33 |
pabelanger | corvus: yah, sorry. kids got at keyboard. Didn't lock it | 18:33 |
corvus | mordred: good point | 18:33 |
mordred | corvus: and in general post is designed so that all post playbooks get run regardless of status of previous one | 18:34 |
corvus | pabelanger: glad you're okay! | 18:34 |
corvus | clarkb: ^ fyi post | 18:34 |
clarkb | mordred: corvus one problem with that approach is when one fails chances are high the rest will too | 18:34 |
clarkb | so now you end up just hitting the timeout N times | 18:34 |
clarkb | but may be it pays to do our best to log anything in post | 18:35 |
mordred | clarkb: I think we should do whatever we can do to ensure we do our best to upload logs no matter what | 18:36 |
mordred | cause a job with missing logs is one of the more frustrating things | 18:36 |
corvus | yeah, i mean, timeout is kind of a blunt instrument. it's not going to be optimal for the "ansible+ssh sits there forever" case. we really just need ansible+ssh to be better about that. | 18:36 |
corvus | the good news is that most post tasks aren't going to run of the main playbook has already timed out | 18:37 |
corvus | so generally, if we're already in that situation, we shouldn't have too many things to run. | 18:38 |
mordred | ++ | 18:40 |
Shrews | could we invent a new thing for 'post-logs'? | 18:41 |
Shrews | maybe that muddies the water a bit though | 18:41 |
*** harlowja has quit IRC | 18:44 | |
clarkb | currently it appears that if you don't set a timeout we don't have a timeout at all. Do we want post-timeout to act the same way? default is don't timeout at all? | 18:44 |
*** harlowja has joined #zuul | 18:48 | |
*** weshay is now known as weshay|bbiab | 18:50 | |
corvus | clarkb: that seems reasonable | 18:54 |
openstackgerrit | Clark Boylan proposed openstack-infra/zuul master: Add post-timeout setting https://review.openstack.org/545439 | 19:02 |
clarkb | thats a rough first draft. I need to figure out how to test that properly. But should give people an idea of what the behavior I think we should have is (so we can determine if that behavior makes sense) | 19:03 |
*** harlowja has quit IRC | 19:10 | |
corvus | clarkb: i think TestAnsible.test_playbook has a timeout test | 19:10 |
openstackgerrit | James E. Blair proposed openstack-infra/zuul master: Make NodeSetParser a regular class https://review.openstack.org/545148 | 19:22 |
openstackgerrit | James E. Blair proposed openstack-infra/zuul master: Make SecretParser a regular class https://review.openstack.org/545149 | 19:22 |
openstackgerrit | James E. Blair proposed openstack-infra/zuul master: Make JobParser a regular class https://review.openstack.org/545150 | 19:22 |
openstackgerrit | James E. Blair proposed openstack-infra/zuul master: Make PipelineParser a regular class https://review.openstack.org/545151 | 19:22 |
openstackgerrit | James E. Blair proposed openstack-infra/zuul master: Make SemaphoreParser a regular class https://review.openstack.org/545152 | 19:23 |
openstackgerrit | James E. Blair proposed openstack-infra/zuul master: Make TenantParser a regular class https://review.openstack.org/545153 | 19:23 |
openstackgerrit | James E. Blair proposed openstack-infra/zuul master: Add ConfigLoader initializer https://review.openstack.org/545180 | 19:23 |
openstackgerrit | James E. Blair proposed openstack-infra/zuul master: Remove tenant argument from UnparsedTenantConfig https://review.openstack.org/545181 | 19:23 |
pabelanger | clarkb: dmsimard: fungi: should we just delete the old volumes now for logs.o.o? We haven't copied over any data, and I think we're pretty much at 30days now | 19:25 |
pabelanger | they are still attached to temp logs server | 19:26 |
fungi | pabelanger: my vote is yes. at this point they're probably almost all past their use-by date | 19:27 |
fungi | also, you probably meant to ask that in #openstack-infra | 19:28 |
pabelanger | doh, yah. asked in wrong room | 19:28 |
pabelanger | apologies | 19:28 |
*** weshay|bbiab is now known as weshay | 19:50 | |
*** harlowja has joined #zuul | 19:50 | |
openstackgerrit | James E. Blair proposed openstack-infra/zuul master: Make JobParser a regular class https://review.openstack.org/545150 | 20:26 |
openstackgerrit | James E. Blair proposed openstack-infra/zuul master: Make PipelineParser a regular class https://review.openstack.org/545151 | 20:26 |
openstackgerrit | James E. Blair proposed openstack-infra/zuul master: Make SemaphoreParser a regular class https://review.openstack.org/545152 | 20:26 |
openstackgerrit | James E. Blair proposed openstack-infra/zuul master: Make TenantParser a regular class https://review.openstack.org/545153 | 20:26 |
openstackgerrit | James E. Blair proposed openstack-infra/zuul master: Add ConfigLoader initializer https://review.openstack.org/545180 | 20:26 |
openstackgerrit | James E. Blair proposed openstack-infra/zuul master: Remove tenant argument from UnparsedTenantConfig https://review.openstack.org/545181 | 20:26 |
openstackgerrit | James E. Blair proposed openstack-infra/zuul master: Add ParseContext class https://review.openstack.org/545448 | 20:26 |
openstackgerrit | James E. Blair proposed openstack-infra/zuul master: Fix typo in PragmaParser https://review.openstack.org/545449 | 20:32 |
openstackgerrit | Merged openstack-infra/zuul master: Make NodeSetParser a regular class https://review.openstack.org/545148 | 20:32 |
openstackgerrit | Merged openstack-infra/zuul master: Make SecretParser a regular class https://review.openstack.org/545149 | 20:32 |
openstackgerrit | Clark Boylan proposed openstack-infra/zuul master: Add post-timeout setting https://review.openstack.org/545439 | 20:43 |
clarkb | corvus: ^ test_playbook was indeed 95% of the way there so I just updated it to check this new stuff | 20:43 |
corvus | clarkb: lgtm | 20:54 |
clarkb | and zuul +1'd it too woot | 21:02 |
pabelanger | looking | 21:03 |
clarkb | I think for ^ we will have to update the scheduler first then restart the clients | 21:18 |
clarkb | because we need to make sure there is a post_timeout value in the executor client args data before the executor server reads it | 21:18 |
openstackgerrit | Merged openstack-infra/zuul master: Add post-timeout setting https://review.openstack.org/545439 | 21:23 |
openstackgerrit | Merged openstack-infra/zuul master: Add host/group vars https://review.openstack.org/544562 | 21:27 |
*** hashar has quit IRC | 21:47 | |
*** JasonCL has quit IRC | 22:01 | |
*** JasonCL has joined #zuul | 22:08 | |
*** rlandy|rover has quit IRC | 22:13 | |
*** JasonCL has quit IRC | 22:17 | |
*** JasonCL has joined #zuul | 22:21 | |
*** JasonCL has quit IRC | 22:30 | |
*** JasonCL has joined #zuul | 22:42 | |
*** JasonCL has quit IRC | 22:47 | |
*** JasonCL has joined #zuul | 23:03 | |
openstackgerrit | James E. Blair proposed openstack-infra/zuul master: Make PipelineParser a regular class https://review.openstack.org/545151 | 23:07 |
openstackgerrit | James E. Blair proposed openstack-infra/zuul master: Make SemaphoreParser a regular class https://review.openstack.org/545152 | 23:07 |
openstackgerrit | James E. Blair proposed openstack-infra/zuul master: Make TenantParser a regular class https://review.openstack.org/545153 | 23:07 |
openstackgerrit | James E. Blair proposed openstack-infra/zuul master: Add ConfigLoader initializer https://review.openstack.org/545180 | 23:07 |
openstackgerrit | James E. Blair proposed openstack-infra/zuul master: Remove tenant argument from UnparsedTenantConfig https://review.openstack.org/545181 | 23:07 |
openstackgerrit | James E. Blair proposed openstack-infra/zuul master: Add ParseContext class https://review.openstack.org/545448 | 23:07 |
openstackgerrit | James E. Blair proposed openstack-infra/zuul master: Fix typo in PragmaParser https://review.openstack.org/545449 | 23:07 |
openstackgerrit | James E. Blair proposed openstack-infra/zuul master: Remove project.unparsed_config https://review.openstack.org/545485 | 23:07 |
corvus | that last change is a memory-use reduction. in my local test, it reduces use by about 50% after several reconfigurations. i still have more changes planned. | 23:09 |
clarkb | nice | 23:10 |
pabelanger | wow | 23:10 |
clarkb | guess I should try and review that stack again | 23:10 |
corvus | i ran pep8, test_model, and test_jobs_executed locally, so i think they should pass | 23:11 |
*** JasonCL has quit IRC | 23:12 | |
*** weshay is now known as weshay_PTO | 23:14 | |
*** JasonCL has joined #zuul | 23:14 | |
*** JasonCL has quit IRC | 23:19 | |
fungi | woah, 50% memory reduction on reconfigs?!? | 23:32 |
clarkb | corvus: supposedly https://review.openstack.org/#/c/545150/4 cannot merge due to a path change? | 23:33 |
clarkb | zuul was happy with it thoug | 23:33 |
clarkb | oh also it will need a rebase for my new timeout I think? | 23:34 |
clarkb | https://review.openstack.org/#/c/545150/4/zuul/configloader.py the code around line 636 there was updated to add post-timeout | 23:35 |
openstackgerrit | James E. Blair proposed openstack-infra/zuul master: Make JobParser a regular class https://review.openstack.org/545150 | 23:50 |
openstackgerrit | James E. Blair proposed openstack-infra/zuul master: Make PipelineParser a regular class https://review.openstack.org/545151 | 23:50 |
openstackgerrit | James E. Blair proposed openstack-infra/zuul master: Make SemaphoreParser a regular class https://review.openstack.org/545152 | 23:50 |
openstackgerrit | James E. Blair proposed openstack-infra/zuul master: Make TenantParser a regular class https://review.openstack.org/545153 | 23:50 |
openstackgerrit | James E. Blair proposed openstack-infra/zuul master: Add ConfigLoader initializer https://review.openstack.org/545180 | 23:50 |
openstackgerrit | James E. Blair proposed openstack-infra/zuul master: Remove tenant argument from UnparsedTenantConfig https://review.openstack.org/545181 | 23:50 |
openstackgerrit | James E. Blair proposed openstack-infra/zuul master: Add ParseContext class https://review.openstack.org/545448 | 23:50 |
openstackgerrit | James E. Blair proposed openstack-infra/zuul master: Fix typo in PragmaParser https://review.openstack.org/545449 | 23:50 |
openstackgerrit | James E. Blair proposed openstack-infra/zuul master: Remove project.unparsed_config https://review.openstack.org/545485 | 23:50 |
*** JasonCL has joined #zuul | 23:53 | |
*** JasonCL has quit IRC | 23:58 |
Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!