Tuesday, 2019-08-20

*** mattw4 has quit IRC00:18
*** michael-beaver has quit IRC00:19
*** jamesmcarthur has quit IRC00:23
*** sgw has quit IRC00:37
*** noorul has joined #zuul00:40
noorulofosos: Looks like canMerge interface got changed in master. It takes extra optional event parameter.00:41
*** altlogbot_1 has quit IRC00:46
*** altlogbot_1 has joined #zuul00:48
noorulI am wondering how tests are not finding this out in https://review.opendev.org/#/c/677257/00:49
*** igordc has quit IRC01:11
noorulcorvus: hi01:51
noorulcorvus: I am not able to get new log view page01:51
noorulcorvus: I added  <IfModule mod_headers.c>01:51
noorul   Header set Access-Control-Allow-Origin "*"01:51
noorul </IfModule>01:51
noorulto httpd.conf in examples folder01:52
noorulofosos: It looks like still the merge is failing02:13
noorulofosos: http://paste.openstack.org/show/760142/02:13
*** noorul has quit IRC02:31
*** noorul has joined #zuul03:00
*** spsurya has joined #zuul03:03
*** bhavikdbavishi has joined #zuul03:08
*** bhavikdbavishi1 has joined #zuul03:11
*** bhavikdbavishi has quit IRC03:13
*** bhavikdbavishi1 is now known as bhavikdbavishi03:13
*** noorul has quit IRC03:47
*** rlandy|ruck|bbl is now known as rlandy|ruck03:56
*** rlandy|ruck has quit IRC04:02
*** raukadah is now known as chkumar|rover05:15
*** sanjayu_ has joined #zuul05:32
*** persia has quit IRC05:49
*** sgw has joined #zuul05:52
openstackgerritMerged zuul/zuul master: web: link the buildset page from the build  https://review.opendev.org/67549305:57
*** persia has joined #zuul06:00
*** persia has quit IRC06:03
*** persia has joined #zuul06:05
*** persia has quit IRC06:05
*** persia has joined #zuul06:05
*** themroc has joined #zuul06:40
*** jpena|off is now known as jpena07:41
*** ianychoi has quit IRC07:46
*** bjackman has joined #zuul07:47
*** ianychoi has joined #zuul07:47
ofososnoorul: this can happen, it is usually some misconfiguration07:59
ofososnoorul: this can also happen, when the pr is changed, between the build and the merge. This is what should happen.08:01
*** hashar has joined #zuul08:08
*** jangutter has joined #zuul08:16
*** panda has quit IRC08:18
*** panda has joined #zuul08:20
ofososnoorul: merging works on our installation. Not sure why it doesn't work for you.08:27
ofososThe usual reason for this to happen is a sneak push, that happens between gate and merge.08:28
*** jamesmcarthur has joined #zuul08:28
*** yolanda has quit IRC08:30
ofososnoorul: Also pr_version=0 looks really weird, that should never happen if you have an approve workflow08:30
ofososDid one of the merges succeed?08:32
*** sgw has quit IRC08:36
*** yolanda has joined #zuul08:43
*** jamesmcarthur has quit IRC08:57
*** jamesmcarthur has joined #zuul08:59
*** jamesmcarthur has quit IRC09:08
*** jamesmcarthur has joined #zuul09:09
*** bhavikdbavishi has quit IRC09:27
*** jamesmcarthur has quit IRC09:41
*** sshnaidm|afk is now known as sshnaidm09:49
openstackgerritSorin Sbarnea proposed zuul/zuul-jobs master: DNM: experiment ansible-version: 2.8  https://review.opendev.org/67744010:11
*** jamesmcarthur has joined #zuul10:16
*** jamesmcarthur has quit IRC10:23
*** aluria has quit IRC10:42
*** jamesmcarthur has joined #zuul10:55
*** jamesmcarthur has quit IRC11:00
*** bjackman has quit IRC11:22
*** jamesmcarthur has joined #zuul11:36
*** jamesmcarthur has quit IRC11:40
*** jamesmcarthur has joined #zuul11:43
*** bhavikdbavishi has joined #zuul11:51
*** bhavikdbavishi1 has joined #zuul11:54
*** bhavikdbavishi has quit IRC11:55
*** bhavikdbavishi1 is now known as bhavikdbavishi11:55
*** rlandy has joined #zuul12:02
*** rlandy is now known as rlandy|ruck12:03
*** hashar has quit IRC12:16
*** hashar has joined #zuul12:19
*** jamesmcarthur has quit IRC12:24
*** jamesmcarthur has joined #zuul12:25
*** jamesmcarthur has quit IRC12:29
*** jamesmcarthur has joined #zuul12:52
openstackgerritMerged zuul/zuul master: Include ref info in smtp reporter subjects  https://review.opendev.org/67728513:08
openstackgerritTristan Cacqueray proposed zuul/zuul master: WIP: builds: set missing start time for merger failure  https://review.opendev.org/67746613:09
*** hogepodge has joined #zuul13:10
*** bhavikdbavishi has quit IRC13:24
openstackgerritJames E. Blair proposed zuul/zuul master: manager: specify report failure in logs  https://review.opendev.org/67176013:45
*** michael-beaver has joined #zuul14:20
*** jeliu_ has joined #zuul14:27
openstackgerritMerged zuul/zuul master: Switch to review.opendev.org for README.rst  https://review.opendev.org/67652914:38
openstackgerritMerged zuul/zuul master: Use a requests session to simplify auth'd calls  https://review.opendev.org/67051114:44
*** themroc has quit IRC14:45
ofososHow is the Openshift provider supposed to sync with the executor?14:58
*** openstackstatus has quit IRC14:58
*** openstack has joined #zuul15:03
*** ChanServ sets mode: +o openstack15:03
*** mattw4 has joined #zuul15:07
*** chkumar|rover is now known as raukadah15:08
*** electrofelix has joined #zuul15:11
*** noorul has joined #zuul15:15
noorulofosos: hi15:16
*** bhavikdbavishi has joined #zuul15:16
*** bhavikdbavishi1 has joined #zuul15:19
*** bhavikdbavishi has quit IRC15:20
*** bhavikdbavishi1 is now known as bhavikdbavishi15:20
openstackgerritDavid Shrewsbury proposed zuul/zuul master: Store autohold requests in zookeeper  https://review.opendev.org/66111415:20
*** sgw has joined #zuul15:25
*** noorul has quit IRC15:34
*** noorul has joined #zuul15:36
*** dkehn has joined #zuul15:36
ofososnoorul: did one of the merges succeed?15:42
openstackgerritTristan Cacqueray proposed zuul/zuul master: docs: add test jobs howto  https://review.opendev.org/67642415:43
openstackgerritTristan Cacqueray proposed zuul/zuul master: docs: add test jobs howto  https://review.opendev.org/67642415:46
noorulI have two PRs both changing the same file. If we merge them one by15:48
noorul         one then there will be a conflict. But when I approve these PRs in15:48
noorul         parallel, the gate pipeline tests them successfully. I was expecting15:48
noorul         it to fail because there is a merge conflict and the second one in15:48
noorul         the queue should run with first patch merged together.15:48
noorulThis is using bitbucket driver. I am not sure whether a driver has anything to do with this logic15:48
corvusnoorul: if git can resolve the conflicts, that might be okay15:49
noorulofosos: The dependent pipeline is not reporting merge conflict. Looks like it is not merging at all15:49
noorultwo PRs changing line 1 of a file15:49
corvusyeah that shouldn't work15:50
openstackgerritTristan Cacqueray proposed zuul/zuul-jobs master: DNM: test match-on-config-updates for third-party-CI  https://review.opendev.org/67749915:50
noorulcorvus: That is working with bitbucket driver15:50
corvusnoorul: it certainly sounds like it could be a bug and should be investigated15:51
noorulcorvus: on the driver side?15:51
noorulcorvus: where should I look?15:52
corvusnoorul: if i were debugging this, i would start by making a test case using zuul's unit testing system.  that will let you reproduce the problem, iteratively debug it, verify that it's resolved, and prevent future regressions15:54
corvusit takes a long time to run all the tests, i wouldn't recommend that, but it's pretty easy to run a single test15:54
noorulcorvus: hmm15:55
corvusnoorul: tests.unit.test_scheduler.TestScheduler.test_project_merge_conflict  is an example of a unit test that covers that case.  you can adapt it to use the bitbucket driver15:55
noorulcorvus: What is the base OS for development ?16:01
*** jamesmcarthur has quit IRC16:03
clarkbnoorul: I use opensuse, others use ubuntu and debian and fedora16:04
clarkbThe test jobs for zuul itself mostly run on ubuntu and centos iirc16:04
noorulI see python3.5 in tox.ini16:04
*** jamesmcarthur has joined #zuul16:05
noorulLatest Ubuntu has 3.6 I think16:05
clarkbya it runs against python36 and 37 too (I have 37 locally)16:06
noorulclarkb: oh okay16:06
clarkbwe could update the envlist to be more specific about that then tell tox not finding an interpreter version is ok16:07
openstackgerritTristan Cacqueray proposed zuul/nodepool master: openstack: handle safely invalid network name  https://review.opendev.org/67750116:07
*** mattw4 has quit IRC16:08
*** sanjayu__ has joined #zuul16:10
openstackgerritDavid Shrewsbury proposed zuul/zuul master: Store autohold requests in zookeeper  https://review.opendev.org/66111416:13
*** sanjayu_ has quit IRC16:13
*** jpena is now known as jpena|off16:14
Shrewstfw you realize your tests fail locally b/c of your running vpn16:16
Shrews:(16:16
Shrewsapparently security does not coexist with stability  :)16:17
fungiit's an old saw that security and convenience define two ends of a spectrum. maybe with stability in there, three points form a plane16:17
fungisecure, convenient, stable... pick one ;)16:17
Shrewsfungi: i pick drinking... is that one?16:18
fungioh, now you've just defined a three-dimensional field16:19
corvusnoorul: can you paste your gate pipeline config?16:19
*** noorul has quit IRC16:20
*** noorul has joined #zuul16:26
corvusnoorul: not sure if you saw my last question -- can you paste your gate pipeline config?16:27
openstackgerritJames E. Blair proposed zuul/zuul master: DNM: sample conversion of gerrit to bitbucket job  https://review.opendev.org/67751016:29
corvusnoorul, ofosos: ^ there's a mock-up of converting that test to bitbucket; indeed, it does not seem to detect the merge conflict.16:30
noorulcorvus: https://etherpad.openstack.org/p/rWL36RmF6W16:30
noorulcorvus: from line 11216:30
openstackgerritJames E. Blair proposed zuul/zuul master: DNM: sample conversion of gerrit to bitbucket job  https://review.opendev.org/67751016:31
corvusnoorul, ofosos: ^ actually, that does work, once i switch the gate pipeline from 'independent' to 'dependent'.  that was a copy/paste from elsewhere in the bitbucket stack -- i'll go leave a review note for that.16:32
corvusnoorul: your pipeline config looks good, and at least a trivial attempt to reproduce this with a unit test isn't showing the error.  so i think you may need to dig into what's happening in your actual environment.16:33
noorulLet me raise two more PRs and share the logs16:34
*** hwangbo has joined #zuul16:35
*** spsurya has quit IRC16:36
corvusmordred: take a look at http://zuul.opendev.org/t/openstack/build/99d3f0cd807b4142a359a101ddb5b8ea/console#1/0/7/localhost  that comes from http://zuul.opendev.org/t/openstack/build/99d3f0cd807b4142a359a101ddb5b8ea/log/job-output.json#870-88516:44
corvusmordred: it looks like if we skip an include_role, we don't have any information about what the role was16:45
corvusmordred: do you think we can/should add that to the json output?16:45
corvusclarkb: ^ fyi16:45
noorulcorvus: scheduler: http://paste.openstack.org/show/760321/16:48
noorulcorvus: executor: http://paste.openstack.org/show/760322/16:49
openstackgerritJeff Liu proposed zuul/zuul-operator master: Add PerconaXDB Cluster to Zuul-Operator  https://review.opendev.org/67731516:49
noorulofosos: Are you able to see merge conflict happening in your environment ?16:50
corvusnoorul: i see entries for the check pipeline -- we're talking about gate, right?16:50
noorulYes16:50
corvusnoorul: observing the behavior in check won't help identify the problem since check pipelines are independent (zuul has no way of knowing when changes may be approved and therefore conflict in the future).  the issue won't show up until you approve both PRs and they are enqueued into gate.16:52
noorulcorvus: I think paste is not allowing more than some number of lines16:53
corvusnoorul: yes, you might want to paste the scheduler logs starting with when you approved the prs16:54
corvusnoorul: there's a length limitation (bytes rather than lines)16:54
noorulcorvus: https://www.dropbox.com/s/168t81enxnpt9hm/executor_log_pruned.txt?dl=016:55
noorulcorvus: https://www.dropbox.com/s/4s0vg9ra36g1cjn/scheduler_log_pruned.txt?dl=016:55
*** hashar has quit IRC16:57
logan-having an odd issue running a cross-repo post job http://paste.openstack.org/raw/760325/ "Project X is not allowed to run job Y" despite job Y including project X in its allowed projects.17:04
*** jamesmcarthur has quit IRC17:04
fungidoes the job use secrets?17:05
logan-ah yes, there is a parent job a few jobs up the chain of inheritance which uses secrets and does not have allowed-projects updated. i'm sure that's it.. thank you17:07
logan-yep -- works now :) thanks fungi17:10
fungibeware that allowing other projects to use jobs with secrets is risky, at least in cases where the two projects in question are under the control of different parties17:11
fungi(unless those secrets are in trusted config projects, where speculative execution will be disallowed)17:11
noorulDo you know any other users other than those mentioned at https://zuul-ci.org/users.html  ?17:13
clarkbnoorul: ansible is using it, ofosos said they were going to production soon. Not sure if flaper87 has elasticsearch on it yet but was looking at it. I think armstrongs is also about to go to production?17:16
Shrewseasystack does "something" with it17:17
noorulcool17:17
funginoorul: also https://superuser.openstack.org/articles/zuul-case-study-tungsten-fabric/17:18
fungiif somebody gets time to summarize that one we can add it to the zuul site too17:20
noorulcorvus: I could not find any error in the logs17:22
funginoorul: oh, also same for https://superuser.openstack.org/articles/continuous-integration-wanted-how-a-large-online-classified-company-uses-zuul/17:22
*** mattw4 has joined #zuul17:24
*** bhavikdbavishi has quit IRC17:24
*** bhavikdbavishi has joined #zuul17:27
ofososnoorul: im not seeing any merge conflicts. But we're not using the dependent pipeline. I think I know what the problem is. I'll push a fix tomorrow. I'm off for today17:27
noorulofosos: ok17:28
fungi"tune in tomorrow for the exciting conclusion"17:30
fungicliffhangers. always leave 'em in suspense17:30
ofososfungi: sorry, beers17:32
fungiofosos: you and i share the same sense of priorities! ;)17:32
*** jamesmcarthur has joined #zuul17:38
AJaegerzuul experts, any idea why we see CONFIG_ERROR for periodic requirements jobs? See http://zuul.opendev.org/t/openstack/buildsets?result=CONFIG_ERROR&pipeline=periodic - you might need to look at the log files, I could not get any further information out of this.17:41
clarkbAJaeger: probably related to our chagne to add ref info to the subject of the smtp reporter17:43
clarkbI'm guessing change.ref isn't set after all?17:43
clarkbI guess we should revert that change17:44
*** sshnaidm is now known as sshnaidm|afk17:44
fungior add something we know will be present and conditionally use one or the other (zuul.branch maybe when zuul.ref is missing?)17:45
clarkbfungi: I think zuul.ref is the one that is always set then you get additional info like branch or tag if the type of ref can be determined17:45
clarkbI think the problem with periodic jobs is that they are not triggered by an action on a ref, they are just run by a timer17:46
*** sanjayu__ has quit IRC17:46
corvusclarkb: the timer enqueues a model.Ref for each branch head17:46
corvusactually, it should enqueue a model.Branch, which is descended from model.Ref, but it's the ref object that supplies the attrs for formatting17:47
*** igordc has joined #zuul17:48
openstackgerritTristan Cacqueray proposed zuul/zuul-jobs master: doc: note that mirror_info isn't actually implemented  https://review.opendev.org/67752918:04
tristanCnot sure about https://review.opendev.org/677529, but i couldn't find any roles that uses mirror_info18:04
AJaegerspeaking about mirror_info, there's also https://review.opendev.org/675703 for review - corvus, did you see that one?18:05
AJaegertristanC: Yes, it's not implemented yet - still in the design phase18:05
*** jamesmcarthur has quit IRC18:05
corvuswell, it sort of went back to the design phase :)18:06
*** jamesmcarthur has joined #zuul18:06
corvusbut yeah, we can merge tristanC's change until we have the first role using it18:06
tristanCcorvus: or perhaps the mirror.rst shouldn't be included in the published doc?18:07
corvusthat's really hard to do with our jobs18:08
corvuseasiest way to make it correct is to get to work implementing it :)  i'll start on the operating system role after lunch18:11
AJaegercorvus, tristanC, do we need https://review.opendev.org/676836 for swift logging? Or is it wrong to do as questions indiciate? "base: generate the zuul-manifest before ara-report". Should we test this change via base-test first?18:11
*** jamesmcarthur has quit IRC18:12
*** jamesmcarthur has joined #zuul18:13
corvusAJaeger: best way to answer the question for sure is base-test18:13
tristanCAJaeger: i'll WIP that, it seems like ara-report could generate an artifact so that it's accessible from the summary18:14
AJaegertristanC: let me -1 - if we want this, we need it in base-test first...18:17
AJaegerthanks, tristanC and corvus18:17
corvusAJaeger, tristanC: maybe create an empty ara-report directory before running the manifest and ara-generate roles18:21
corvustristanC, AJaeger: another idea to think about -- have the manifest role take a list of directories not to recurse into18:22
corvustristanC, AJaeger: yet another idea -- have the web app ignore ara-report (do it in actions)18:23
* corvus stops brainstorming and afks18:23
openstackgerritMerged zuul/zuul-jobs master: Update documentation for Debian/Ubuntu mirror_info  https://review.opendev.org/67570318:25
*** bjackman has joined #zuul18:29
*** noorul has quit IRC18:32
*** electrofelix has quit IRC18:35
openstackgerritMerged zuul/zuul-jobs master: doc: note that mirror_info isn't actually implemented  https://review.opendev.org/67752918:45
fungifollowing up on the earlier leboncoin case study earlier, someone just pointed this comment out to me: https://twitter.com/gchenuet/status/116386024544320716818:55
fungiseems they're still fans ;)18:56
Shrewscorvus: is this a known random failure? https://ae2197746e23c7b714c7-0b8936c820696cdb6c8467dc1e642b32.ssl.cf1.rackcdn.com/661114/13/check/tox-py36/4a2da8a/testr_results.html.gz18:56
fungitakes a lot of scrolling to get to that traceback18:57
corvusShrews: news to me18:58
fungii haven't seen that one before that i can recall18:58
Shrewstox-py35 passed18:58
*** jamesmcarthur has quit IRC18:58
corvus15 is a lot of builds, so it's probably right on the edge18:59
tristanCcorvus: it seems like "having the manifest role takes a list of directories not to recurse into" is the most efficient improvement19:10
tristanCusing ["ara-report"] as a default value19:10
*** bjackman has quit IRC19:14
*** jamesmcarthur has joined #zuul19:28
openstackgerritTristan Cacqueray proposed zuul/zuul-jobs master: generate-zuul-manifest: add excludes list  https://review.opendev.org/67755619:32
*** jamesmcarthur has quit IRC19:59
*** jamesmcarthur has joined #zuul20:00
openstackgerritTristan Cacqueray proposed zuul/zuul-jobs master: generate-zuul-manifest: add excludes list  https://review.opendev.org/67755620:07
corvusfungi: what was the reason we didn't make 'type' a list in 675703 ?20:34
*** jamesmcarthur has quit IRC20:40
*** jamesmcarthur_ has joined #zuul20:40
fungigood question, i think we certainly could. once i get dinner out of the way i'll double-check that there wasn't a good reason20:41
fungithough frankly, the odds it will be anything other than "deb" are slim since "deb-src" would only be if you wanted to be able to fetch source packages or install build dependencies of source packages20:42
corvusyeah, but we're aiming for future-proofing here :)20:43
corvus*someone* might want that :)20:43
*** jamesmcarthur_ has quit IRC20:47
*** bhavikdbavishi has quit IRC20:47
fungithe debian project might even want to do that ;)20:47
fungibut yeah, i think we ought to be able to switch it for improved consistency20:47
fungiif you need to not have matrix expansion for some set, just make a separate entry20:47
fungii'll whip up a followup change shortly20:48
corvusfungi: don't worry about it -- i'm working on an implementation, i'll include it there20:48
fungiwfm, thanks!20:49
*** jamesmcarthur has joined #zuul20:55
*** jamesmcarthur has quit IRC20:58
*** jamesmcarthur has joined #zuul20:58
*** jamesmcarthur has quit IRC21:04
openstackgerritTristan Cacqueray proposed zuul/zuul-jobs master: generate-zuul-manifest: add excludes list  https://review.opendev.org/67755621:11
corvusfungi: i think we may also need to add something to filter for architectures21:17
corvusfungi: with the current role, on aarch64, we add '-ports' to the url21:17
corvushrm... i guess we could do the same here and make it automatic21:18
corvusor we wanted it to be explicit, we could add it to the mirror_info variable...21:18
fungiyeah, at least for debian the ports tree is independent and could be on different servers even21:20
fungii don't know that i would break up the url at all, easier to treat it as a single string21:21
fungii'd rather not make assumptions about how someone's mirror is organized21:22
corvusfungi: so are you voting for adding "architectures: ['amd64', 'aarch65']" to the list?21:22
fungier21:23
fungii'm slightly confused by the example21:23
corvusmodulo typos :)21:23
fungi['amd64', 'arm64'] i guess... let me check21:24
corvusfungi: it would be a list of architectures hosted at the url specified in the list, then the role would include that list item iff the current architecture matched one in the list21:24
funginormally you wouldn't do cross/multi-arch sources lists21:24
fungioh, i see21:24
corvusif it's not possible (or unfeasible) we could make that a singleton rather than list21:25
fungibecause you'll have systems for each architecture which need different urls21:25
corvusthe existing values of ansible_architecture which are relevant seem to be 'x86_64' and 'aarch64'21:26
fungigot it, which will somewhere be translated to amd64 and arm6421:26
fungior not translated at all, simply used by ansible for selecting?21:27
corvusi don't think we have to write the arch into the config file, so i think it's really 2 separate things21:27
corvusyeah that21:27
corvusansible_architecture for selecting, then apt handles the rest21:27
fungiso yeah, maybe it needs to be the ansible architectures, not the debian architectures21:27
fungiwhere it likely gets hairy is if you want to cross-build and install packages for one architecture on a machine of a different architecture, then you'll need to translate from debian architecture names to ansible architecture names21:29
fungibut i'm happy to consider that a corner case21:29
fungi(the amd64/arm64 example is not really a great one, since both are official architectures for debian now)21:30
corvusi guess you could say "include the arm64 sources entries on this amd64 host" that should work right?21:31
fungiin opendev's case the url for both would be identical21:31
fungibut yeah21:31
corvusreally the only reason not to include them all is just so we're not bulking up the package list on our x86 hosts21:31
fungito include the arm64 sources entries you'd need to know to look up the aarch64 set21:31
funginot a big deal21:31
corvusright, but with the system we're talking about, the admin could say "architectures: ['x86_64']" for a url which hosts arm packages and get the full set21:32
fungiwell, every additional entry is another (set of) http gets and possibly gpg signature validations and index additions and so on... so increases the time/workload unnecessarily21:33
fungi(as the reason not to include them all always)21:33
corvusyeah, that seems to make this worth doing21:33
fungiagreed21:33
corvusokay, i'll do it like that, thanks :)21:34
fungithank *you*!21:34
openstackgerritTristan Cacqueray proposed zuul/zuul-jobs master: generate-zuul-manifest: add excludes list  https://review.opendev.org/67755621:36
*** mattw4 has quit IRC21:43
*** mattw4 has joined #zuul21:51
*** mattw4 has quit IRC22:02
*** mattw4 has joined #zuul22:07
*** zbr has quit IRC22:14
*** zbr has joined #zuul22:15
*** armstrongs has joined #zuul22:18
*** armstrongs has quit IRC22:29
openstackgerritJames E. Blair proposed zuul/zuul-jobs master: Add configure-os-mirrors role  https://review.opendev.org/67757822:37
corvusfungi, tristanC, clarkb, AJaeger: ^22:37
corvusi'm hoping the rest will be much simpler than that22:38
*** jeliu_ has quit IRC22:40
clarkbcorvus: left some comments on that, but it seems to be fairly compelte for the distro pacakge repo mirrors22:54
*** mattw4 has quit IRC23:03
*** zbr has quit IRC23:22
*** zbr has joined #zuul23:23
openstackgerritIan Wienand proposed zuul/zuul-jobs master: upload-logs-swift: fix keystoneauth1 exceptions  https://review.opendev.org/67758423:57

Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!