| rlandy | weshay|rover: ok - have a local run going - will cat out the log file | 00:28 |
|---|---|---|
| rlandy | rfolco: weshay|rover: that is all that shows up in the promoter file http://pastebin.test.redhat.com/799915 | 00:38 |
| rlandy | test does nothing | 00:38 |
| rlandy | imho, the server is wrong | 00:38 |
| rlandy | that's localhost:5000 to localhost:50504 | 00:38 |
| EmilienM | oops | 00:50 |
| weshay|rover | rlandy, ugh.. ok.. let's chat tomorrow | 01:51 |
| *** pierrepr1netti has joined #oooq | 01:52 | |
| *** pierreprinetti has quit IRC | 01:56 | |
| *** saneax has joined #oooq | 02:08 | |
| *** weshay|rover has quit IRC | 02:20 | |
| *** rfolco has quit IRC | 02:27 | |
| *** dsneddon has quit IRC | 02:29 | |
| *** rlandy has quit IRC | 02:33 | |
| *** saneax has quit IRC | 02:40 | |
| *** ykarel has joined #oooq | 03:32 | |
| *** skramaja has joined #oooq | 04:05 | |
| *** ykarel has quit IRC | 04:24 | |
| *** ratailor has joined #oooq | 04:43 | |
| *** ykarel has joined #oooq | 04:55 | |
| *** holser has joined #oooq | 04:57 | |
| *** ratailor has quit IRC | 05:00 | |
| *** udesale has joined #oooq | 05:00 | |
| *** brault has quit IRC | 05:12 | |
| *** brault has joined #oooq | 05:15 | |
| *** ratailor has joined #oooq | 05:26 | |
| *** jtomasek has joined #oooq | 05:30 | |
| *** dsneddon has joined #oooq | 05:41 | |
| *** akahat has joined #oooq | 05:46 | |
| *** dsneddon has quit IRC | 05:46 | |
| *** holser has quit IRC | 05:53 | |
| *** marios has joined #oooq | 05:59 | |
| *** jfrancoa has joined #oooq | 06:00 | |
| *** jfrancoa has quit IRC | 06:04 | |
| *** kopecmartin|off is now known as kopecmartin | 06:05 | |
| *** surpatil has joined #oooq | 06:05 | |
| *** saneax has joined #oooq | 06:10 | |
| *** brault has quit IRC | 06:17 | |
| *** yolanda has quit IRC | 06:34 | |
| *** tosky has joined #oooq | 06:53 | |
| marios | arxcruz|rover: zbr|ruck any idea if things are borked looked at grafana right now didn't see a lot of fails (asking if this is a nit in the code or if there is general problem with ci @ https://review.opendev.org/#/c/683270/4 | 07:02 |
| marios | arxcruz|rover: zbr|ruck i suspect a nit but asking in case | 07:02 |
| *** soniya29 has joined #oooq | 07:05 | |
| *** tesseract has joined #oooq | 07:07 | |
| *** chem has joined #oooq | 07:24 | |
| marios | arxcruz|rover: zbr|ruck is that one known http://zuul.openstack.org/builds?job_name=tripleo-ci-centos-7-containers-multinode fails in check last few hours | 07:28 |
| marios | arxcruz|rover: zbr|ruck https://etherpad.openstack.org/p/ruckroversprint16 don't see it there... going to try a recheck (i came to harrass you from https://review.opendev.org/#/c/681669/3 | 07:30 |
| *** bogdando has joined #oooq | 07:31 | |
| *** akahat has quit IRC | 07:36 | |
| *** brault has joined #oooq | 07:38 | |
| zbr|ruck | monring! | 07:39 |
| *** holser has joined #oooq | 07:40 | |
| *** recheck has quit IRC | 07:41 | |
| *** recheck has joined #oooq | 07:41 | |
| zbr|ruck | just returned, so I need it of time to catch-up with irc/email | 07:41 |
| *** brault has quit IRC | 07:43 | |
| *** amoralej|off is now known as amoralej | 07:44 | |
| zbr|ruck | re errors, i remember from last week that timeouts were a real issue, but these are purely caused by too long deployment | 07:52 |
| zbr|ruck | not caused by read infra issues, is just tripleo performance | 07:52 |
| zbr|ruck | they went abobe the limit of 3.5h | 07:52 |
| *** jfrancoa has joined #oooq | 07:53 | |
| zbr|ruck | the only thing I can do for the moment is to make that job non-voting. | 07:55 |
| marios | zbr|ruck: ack lets see with recheck for now | 07:55 |
| zbr|ruck | but I am not sure if I can make that call | 07:55 |
| marios | zbr|ruck: but it might be a legit issue though there was one green one so... not sure | 07:55 |
| zbr|ruck | marios: it was a known issue on Tuesday, i doubt you will suceed. | 07:55 |
| zbr|ruck | you may suceed on one, but how about the "big picture"? | 07:56 |
| zbr|ruck | not to mention what is the outcome if everyone does rechecks | 07:56 |
| marios | sshnaidm: ksambor was asking me about http://logs.rdoproject.org/45/680345/26/openstack-check/tripleo-ci-centos-7-ovb-3ctlr_1comp_1supp-featureset039/e39ad1f/logs/ can the zuul repro do ovb jobs i recall you were looking at that or i mis remember | 07:57 |
| marios | ksambor: i assume that is what you mean by 'non deprecated reproducer' like the zuul reproducer | 07:57 |
| marios | http://logs.rdoproject.org/45/680345/26/openstack-check/tripleo-ci-centos-7-ovb-3ctlr_1comp_1supp-featureset039/e39ad1f/logs/README-reproducer.html that | 07:57 |
| sshnaidm | marios, yes, of course | 07:57 |
| zbr|ruck | panda: sshnaidm ; what do you think about the ^ timeouts issue? any recommandation, as in first steps. | 07:57 |
| marios | sshnaidm: :D ack thanks ksambor there you go | 07:57 |
| ksambor | sshnaidm, marios: thanks! | 07:58 |
| sshnaidm | zbr|ruck, everything is worse | 07:58 |
| sshnaidm | zbr|ruck, containers prep time takes a long: https://snapshot.raintank.io/dashboard/snapshot/SjZ16RtMXZCFbEIOjgZ7N7KKJvOpKBhB?orgId=2 | 07:59 |
| panda | is the promoter server working now ? | 08:04 |
| panda | nope it doesn't | 08:05 |
| panda | oh yes it does | 08:06 |
| zbr|ruck | where do I have to raise a ticket if rdo kibana is down? osci? | 08:07 |
| panda | was it ever up ? | 08:07 |
| zbr|ruck | it was at some point, | 08:08 |
| *** ykarel is now known as ykarel|lunch | 08:08 | |
| *** apetrich has quit IRC | 08:12 | |
| *** apetrich has joined #oooq | 08:22 | |
| zbr|ruck | arxcruz|rover: do you know what can cause permission defined on /root/DLRN: see http://logstash.openstack.org/#/dashboard/file/logstash.json?query=message:%5C%22Permission%20denied:%20'%2Froot%2FDLRN'%5C%22 | 08:24 |
| *** derekh has joined #oooq | 08:28 | |
| *** jpena|off is now known as jpena | 08:30 | |
| *** jbadiapa has joined #oooq | 08:35 | |
| *** akahat has joined #oooq | 08:36 | |
| *** jbadiapa has quit IRC | 08:37 | |
| *** jbadiapa has joined #oooq | 08:38 | |
| *** jbadiapa has quit IRC | 08:42 | |
| *** holser has quit IRC | 08:43 | |
| *** akahat has quit IRC | 08:43 | |
| *** akahat has joined #oooq | 08:43 | |
| *** holser has joined #oooq | 08:44 | |
| *** ykarel|lunch is now known as ykarel | 08:46 | |
| *** ccamacho has joined #oooq | 09:01 | |
| *** brault has joined #oooq | 09:05 | |
| *** brault has quit IRC | 09:09 | |
| amoralej | zbr|ruck, i've just reported https://bugs.launchpad.net/tripleo/+bug/1845166 and set as promotion-blocker | 09:18 |
| openstack | Launchpad bug 1845166 in tripleo "[queens] Periodic OVB jobs failing in OC deploy when trying to ssh to nodes" [Undecided,New] | 09:18 |
| amoralej | non-ovb are now passing after the patch to not bindmount /usr/bin and /usr/sbin | 09:18 |
| amoralej | but ovb are still failing | 09:19 |
| zbr|ruck | amoralej: thaks, added to https://etherpad.openstack.org/p/ruckroversprint16 | 09:19 |
| arxcruz|rover | zbr|ruck: hello, sorry, no, just wakeup, stay until late yesterday... | 09:21 |
| *** ratailor_ has joined #oooq | 09:26 | |
| *** ratailor has quit IRC | 09:28 | |
| *** saneax is now known as saneax|AFK | 09:34 | |
| zbr|ruck | arxcruz|rover or sshnaidm : any hints around this ovb failure https://bugs.launchpad.net/tripleo/+bug/1845166 ? | 09:35 |
| openstack | Launchpad bug 1845166 in tripleo "[queens] Periodic OVB jobs failing in OC deploy when trying to ssh to nodes" [Undecided,New] | 09:35 |
| arxcruz|rover | zbr|ruck: that's the 1 milion dollar question :) | 09:36 |
| arxcruz|rover | dollars* | 09:36 |
| *** saneax|AFK has quit IRC | 09:40 | |
| *** brault has joined #oooq | 09:42 | |
| *** brault has quit IRC | 09:42 | |
| *** brault has joined #oooq | 09:42 | |
| arxcruz|rover | oh boy... | 09:50 |
| arxcruz|rover | amoralej: ovb still failing in the same issue ? | 09:51 |
| arxcruz|rover | amoralej: also, what about https://trello.com/c/DvgaBFim/1104-cixlp1843259tripleociproa-periodic-rocky-fs020-job-fails-tempest-tests-tempestscenariotestsecuritygroupsbasicopstestsecuritygrou?menu=filter&filter=label:TripleO-rocky ? | 09:52 |
| arxcruz|rover | didn't get that link of auto-releases | 09:53 |
| amoralej | arxcruz|rover, no idea | 09:59 |
| amoralej | that was accidental attachment | 10:00 |
| arxcruz|rover | amoralej: no idea of what? ovb or trello link ? | 10:00 |
| arxcruz|rover | :D | 10:00 |
| amoralej | trello link | 10:00 |
| arxcruz|rover | ok | 10:00 |
| amoralej | trello has some magic key combinations | 10:00 |
| arxcruz|rover | amoralej: and the ovb? :D | 10:00 |
| amoralej | probably i missedtype | 10:00 |
| amoralej | ovb still failing in queens | 10:01 |
| amoralej | is that known issue? | 10:01 |
| amoralej | i didn't find it reported | 10:01 |
| *** dsneddon has joined #oooq | 10:09 | |
| *** brault has quit IRC | 10:11 | |
| *** openstackstatus has quit IRC | 10:12 | |
| *** openstack has joined #oooq | 10:14 | |
| *** ChanServ sets mode: +o openstack | 10:14 | |
| *** dtantsur|afk is now known as dtantsur | 10:17 | |
| arxcruz|rover | amoralej: do you have logs ? | 10:38 |
| arxcruz|rover | from the failure ? | 10:38 |
| arxcruz|rover | we merged the dumb-init yesterday | 10:38 |
| arxcruz|rover | not sure if the ran already get it | 10:38 |
| amoralej | arxcruz|rover, yes, the dumb-init is fixed | 10:39 |
| amoralej | those jobs are passing | 10:39 |
| amoralej | this is other blocker | 10:39 |
| amoralej | i think i pasted log links in the lp | 10:39 |
| amoralej | lemme check | 10:39 |
| amoralej | arxcruz|rover, http://logs.rdoproject.org/openstack-periodic-24hr/opendev.org/openstack/tripleo-ci/master/periodic-tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset035-queens/5996bf7/logs/undercloud/home/zuul/overcloud_deploy.log.txt.gz | 10:40 |
| amoralej | arxcruz|rover, you can check ovb jobs failed in last run | 10:42 |
| amoralej | https://review.rdoproject.org/zuul/builds?pipeline=openstack-periodic-24hr | 10:42 |
| amoralej | but non ovb passed | 10:42 |
| arxcruz|rover | amoralej: ack thanks | 10:43 |
| *** weshay has joined #oooq | 10:50 | |
| *** ratailor_ has quit IRC | 10:58 | |
| *** udesale has quit IRC | 11:08 | |
| *** udesale has joined #oooq | 11:09 | |
| arxcruz|rover | dtantsur: https://logs.rdoproject.org/openstack-periodic-master/opendev.org/openstack/tripleo-ci/master/periodic-tripleo-ci-centos-7-ovb-1ctlr_2comp-featureset020-master/41a87ee/logs/undercloud/home/zuul/overcloud_prep_images.log.txt.gz#_2019-09-23_14_56_37 | 11:15 |
| arxcruz|rover | dtantsur: any idea? if it's your fault even better :P | 11:15 |
| dtantsur | arxcruz|rover: it may be mistral or tripleo-common ignoring ironicclient deprecations for too long | 11:16 |
| * dtantsur asks rpittau on #openstack-ironic | 11:16 | |
| arxcruz|rover | weshay: i did not get your email, what do you need with toe cockpit ? | 11:18 |
| weshay | arxcruz|rover, just got in.. thanks | 11:18 |
| weshay | arxcruz|rover, thanks for looking at queens.. I told Scott to push out the release 1 week | 11:19 |
| weshay | so.. you guys have some time | 11:19 |
| *** amoralej is now known as amoralej|lunch | 11:32 | |
| * marios biab | 11:33 | |
| *** marios has quit IRC | 11:33 | |
| *** jpena is now known as jpena|lunch | 11:34 | |
| weshay | arxcruz|rover, zbr|ruck sync up https://meet.google.com/vkf-ouax-prn | 11:39 |
| *** marios has joined #oooq | 11:58 | |
| *** derekh has quit IRC | 12:00 | |
| weshay | arxcruz|rover, zbr|ruck https://review.rdoproject.org/r/#/c/22445/ | 12:06 |
| *** rfolco has joined #oooq | 12:10 | |
| weshay | zbr|ruck, arxcruz|rover https://review.opendev.org/#/c/683425/ | 12:14 |
| weshay | arxcruz|rover, https://e4eefaba0e0f3a2b2287-ba7281a549a418e9d574e043a1b402b5.ssl.cf1.rackcdn.com/683425/1/check/tripleo-ci-centos-7-standalone/b054301/logs/undercloud/home/zuul/standalone_deploy.log.txt.gz | 12:18 |
| weshay | marios, rfolco panda when rlandy gets in.. you guys may want to sync up on https://review.rdoproject.org/r/#/c/22445/ | 12:25 |
| weshay | I am interested to know what kind of code coverage we have on container-push | 12:25 |
| weshay | and what tags are used in testing | 12:25 |
| *** aakarsh has joined #oooq | 12:27 | |
| marios | weshay: ack | 12:30 |
| weshay | rfolco, chandankumar https://meet.google.com/unv-qweu-tdp?authuser=1 | 12:31 |
| rfolco | weshay, sec | 12:33 |
| *** jpena|lunch is now known as jpena | 12:35 | |
| arxcruz|rover | zbr|ruck: i'm doing a live code for the interns on the cockpit at https://meet.google.com/sbh-dtby-bdb?pli=1&authuser=0 if you're interested | 12:37 |
| *** brault has joined #oooq | 12:37 | |
| *** rlandy has joined #oooq | 12:39 | |
| *** brault has quit IRC | 12:40 | |
| rlandy | rfolco: marios: panda: are we meeting today? | 12:40 |
| *** brault has joined #oooq | 12:40 | |
| rlandy | we need to discuss the promotion part of the testing | 12:40 |
| rlandy | weshay wanted us to use that part to test quay work | 12:40 |
| rlandy | http://pastebin.test.redhat.com/799915 is the output of the promoter logs | 12:41 |
| rlandy | it is unclear to me whether anything actually runs here | 12:42 |
| rlandy | also molecule logs are very incomplete | 12:42 |
| marios | rlandy: yes lets sync at community? | 12:42 |
| marios | rlandy: or we could do half hour before... i.e. in 20 mins? | 12:42 |
| marios | rfolco: available? | 12:42 |
| rlandy | marios: I thought the community meeting was taken? | 12:42 |
| rfolco | marios, in bug trage atm | 12:43 |
| rfolco | triage | 12:43 |
| marios | rfolco: k | 12:43 |
| marios | rlandy: we can try after community | 12:44 |
| marios | rlandy: though we have a busy/calls afternoon | 12:45 |
| marios | rlandy: lets see how it goes... | 12:45 |
| rlandy | marios: imho, this is pretty serious | 12:45 |
| *** amoralej|lunch is now known as amoralej | 12:46 | |
| rlandy | let's start at community meeting and see afterwards | 12:46 |
| marios | rlandy: yes | 12:47 |
| chandankumar | rlandy: https://review.rdoproject.org/r/#/c/22529/ | 12:48 |
| rfolco | rlandy, marios panda: I'm free now if you want to start early | 12:48 |
| marios | rfolco: ack lets say in 10 mins rlandy k? | 12:48 |
| marios | sending invite now | 12:49 |
| rfolco | wfm | 12:49 |
| chandankumar | rlandy: we have container-build and standalone working for train | 12:49 |
| *** brault has quit IRC | 12:49 | |
| *** brault has joined #oooq | 12:50 | |
| rlandy | ok | 12:53 |
| rlandy | chandankumar: ok - so do you want to assign me some tasks/cards | 12:54 |
| rlandy | I can do the promotion pipelines | 12:54 |
| *** aakarsh has quit IRC | 12:54 | |
| rlandy | periodic jobs? | 12:54 |
| chandankumar | rlandy: https://review.rdoproject.org/r/#/c/22529/ | 12:54 |
| chandankumar | rlandy: putting notes on the card itself | 12:54 |
| rlandy | I see three jobs on that review | 12:54 |
| rlandy | ok - so we need those reviews in | 12:58 |
| chandankumar | rlandy: https://tree.taiga.io/project/tripleo-ci-board/task/1299 | 12:58 |
| chandankumar | rlandy: assigning one more task | 12:59 |
| rlandy | looking | 12:59 |
| *** jaosorior has quit IRC | 12:59 | |
| marios | rlandy: rfolco joining? | 13:01 |
| chandankumar | rlandy: https://tree.taiga.io/project/tripleo-ci-board/task/1300 | 13:01 |
| rlandy | chandankumar: going into meeting - will take care of those this afternoon | 13:01 |
| rlandy | marios: yes | 13:01 |
| chandankumar | rlandy: we will start with scenrario 1-4 ad other standalone scenario and trigger in this fashion https://review.rdoproject.org/r/#/c/22529/ | 13:02 |
| chandankumar | rlandy: sure,thanks! | 13:02 |
| *** brault has quit IRC | 13:03 | |
| *** brault has joined #oooq | 13:05 | |
| *** brault has quit IRC | 13:10 | |
| *** saneax|AFK has joined #oooq | 13:11 | |
| arxcruz|rover | zbr|ruck: weshay: https://review.rdoproject.org/r/#/c/22533 | 13:30 |
| *** surpatil has quit IRC | 13:30 | |
| weshay | rfolco, the bluejeans button is still visible on the community call | 13:31 |
| rfolco | weshay, its on purpose, we are experimenting this... I'll remove when we decide to keep it | 13:32 |
| panda | marios: rfolco may I know exactly what are you working on right now ? | 13:32 |
| rfolco | panda, sure | 13:32 |
| rfolco | panda, adding logs to the job, and calling image upload w/ staging containers, pulling from local registry:5000 and pushing to local registry:8787 (for both container/manifest) | 13:34 |
| rfolco | panda, I think you can work on the promotion_run end-to-end test (add it to zuul job and check if the promotion really happens) | 13:36 |
| rfolco | panda, can you please answer my questions on https://tree.taiga.io/project/tripleo-ci-board/task/1293 ? | 13:37 |
| *** soniya29 has quit IRC | 13:38 | |
| rfolco | panda, answering your question: working on #1293 and #1282 | 13:44 |
| *** chem has quit IRC | 14:00 | |
| *** Vorrtex has joined #oooq | 14:01 | |
| marios | weshay: that event has bluejeans tbd by the way (rhi) | 14:02 |
| *** dsneddon has joined #oooq | 14:02 | |
| rlandy | chandankumar: I may have to take https://tree.taiga.io/project/tripleo-ci-board/task/1301 to do task 1300 - depending on parenting ... will see ho wit goes | 14:03 |
| chandankumar | rlandy: sure | 14:03 |
| *** dsneddon has quit IRC | 14:07 | |
| mjturek | rfolco: sorry about that forgot to read the topic | 14:10 |
| rfolco | mjturek, np :) | 14:11 |
| mjturek | This is the patch: https: //review.opendev.org/#/c/683997/ I guess the only thing I'm unsure of is why the file doesn't exist | 14:12 |
| mjturek | but I'm pretty sure it wouldn't in the tripleoci case | 14:12 |
| *** dsneddon has joined #oooq | 14:12 | |
| rfolco | mjturek, well, test -f will fail anyway... what does change ? | 14:13 |
| rfolco | mjturek, I don't understand why the file does not exist for ci case | 14:15 |
| mjturek | hmm let me see what I can find then | 14:15 |
| rfolco | mjturek, I guess this is not a real fix, unless I am missing something | 14:15 |
| mjturek | rfolco: fair enough! I will dig deeper | 14:16 |
| *** skramaja has quit IRC | 14:16 | |
| *** chem has joined #oooq | 14:16 | |
| *** dsneddon has quit IRC | 14:17 | |
| *** dsneddon has joined #oooq | 14:17 | |
| mjturek | rfolco: if the file doesn't exist, test would skip the sed | 14:18 |
| mjturek | http://paste.openstack.org/show/779183/ | 14:18 |
| mjturek | fwiw ^ | 14:18 |
| rfolco | but will return 1 | 14:18 |
| rfolco | and will fail | 14:18 |
| rfolco | isn't ? | 14:18 |
| mjturek | ahhh you're right | 14:18 |
| rfolco | do a test -f in a xxxxxxx (non-existing) | 14:18 |
| mjturek | http://paste.openstack.org/show/779184/ | 14:19 |
| mjturek | yep same problem | 14:19 |
| *** dsneddon has quit IRC | 14:22 | |
| rfolco | mjturek, why the file does not exist for ci ? | 14:22 |
| mjturek | the thought was that the repos get overridden | 14:23 |
| mjturek | but I wasn't sure if that meant the file was also deleted | 14:23 |
| mjturek | need to check | 14:23 |
| rlandy | rfolco: why registry registry:8787 rather than registry:5050 - which is what we created in setup? | 14:24 |
| rfolco | rlandy, just an example, we can switch to whatever is in place now | 14:24 |
| rfolco | rlandy, can you answer my questions there ? | 14:25 |
| rlandy | rfolco: sorry - which questions? | 14:25 |
| rfolco | rlandy, :) | 14:25 |
| rfolco | rlandy, why you asking me 8787 ? | 14:25 |
| rlandy | to check if I need to change the setup. | 14:26 |
| rfolco | rlandy, I thought you looked at https://tree.taiga.io/project/tripleo-ci-board/task/1293?kanban-status=1447275 | 14:26 |
| *** brault has joined #oooq | 14:26 | |
| rfolco | rlandy, it can be any port | 14:26 |
| rlandy | rfolco: no - I just want to finish the tests | 14:26 |
| rlandy | and get them in now | 14:26 |
| rlandy | panda: I'd like to chat with you for a few minutes before you log off today re: integrating the tests into the current molecule work | 14:31 |
| rlandy | and how to call them | 14:31 |
| rlandy | I think we could make the tests works by default for staging | 14:31 |
| rlandy | and then add what is left to ensure it runs against the real reproducer afterwards | 14:32 |
| panda | rlandy: I'll be here for another 1:30, then away for a couple of hours, then back again until late. | 14:34 |
| rlandy | panda: so any time that works for you | 14:34 |
| rlandy | later is fine as well | 14:34 |
| rlandy | panda: ping me when you have time | 14:36 |
| marios | panda: rlandy: rfolco http://logs.rdoproject.org/02/22002/52/check/tripleo-ci-promotion-staging/e2eb860/job-output.txt.gz grep "Inspect manifests in localhost:5000" | 14:39 |
| rlandy | nice | 14:39 |
| rfolco | marios, cool | 14:39 |
| rfolco | marios, cool story bro | 14:39 |
| marios | rfolco: indeed .. finally | 14:40 |
| rfolco | marios, do you have a min to chat ? | 14:40 |
| *** kopecmartin is now known as kopecmartin|off | 14:40 | |
| rfolco | marios, its ok if you are shutting down | 14:41 |
| marios | panda: re the molecule route, i updated the stale hash there this morning but it fails on tcp issue http://logs.rdoproject.org/02/22002/36/check/rdo-tox-molecule/e1c704a/tox/reports.html i did not pursue that some more | 14:41 |
| marios | rfolco: i'm listening in the call still (rhi) | 14:41 |
| marios | rfolco: after it? | 14:41 |
| rfolco | oh ok | 14:41 |
| marios | rfolco: should be few | 14:41 |
| rfolco | marios, after I have all dfgs call | 14:41 |
| marios | panda: i was just iterating on the ci-promotion-staging job today | 14:41 |
| marios | rfolco: ah right | 14:41 |
| marios | rfolco: k , tomorrow your morning? | 14:41 |
| marios | rfolco: schedule it maybe just send an invite | 14:42 |
| marios | rfolco: well we might have few mins before the all hands lets see | 14:42 |
| rfolco | marios, will get my questions answered somehow today | 14:42 |
| rfolco | everyone is busy | 14:42 |
| rfolco | I am developing kind of attention deficit | 14:43 |
| panda | marios: I know how to test it in isolation | 14:44 |
| marios | panda: ok ... but we need to have that discussion given the overhead. (like setting up certs etc).. in this case it may make sense to just have the tripleo-ci-promotion-staging job | 14:47 |
| marios | panda: we can talk some more about it tomorrow (will you update the patch or just have an idea?) | 14:48 |
| marios | rfolco: we have 10 mins wana talk now? | 14:50 |
| panda | marios: I havde the manifest push test working in isolation locally | 14:50 |
| marios | panda: ok post it on the review | 14:50 |
| marios | panda: it should not conflict with my stuff which is in staging.yml | 14:51 |
| marios | panda: lets get both molecule job and the promotion staging job green for that and we can discuss on next scrum/sync | 14:52 |
| weshay | arxcruz|rover, ready for me to test your change? | 14:55 |
| weshay | arxcruz|rover, https://review.rdoproject.org/r/#/c/22533/ | 14:55 |
| arxcruz|rover | weshay: yes, but it's returning the seconds | 14:55 |
| arxcruz|rover | not in HH:MM:SS | 14:56 |
| weshay | arxcruz|rover, so I found that seconds.. don't show up correctly in grafana | 14:57 |
| weshay | can you show me? | 14:57 |
| *** akahat has quit IRC | 14:58 | |
| arxcruz|rover | weshay: yes i can | 14:59 |
| arxcruz|rover | let me just restart my laptop | 15:00 |
| weshay | arxcruz|rover, k | 15:00 |
| *** ratailor has joined #oooq | 15:00 | |
| *** tosky has quit IRC | 15:00 | |
| arxcruz|rover | weshay: back | 15:03 |
| weshay | arxcruz|rover, https://meet.google.com/gvk-mgnu-hzn | 15:03 |
| *** ratailor has quit IRC | 15:03 | |
| *** ccamacho has quit IRC | 15:05 | |
| *** Vorrtex has quit IRC | 15:10 | |
| mjturek | rfolco: FYI addressed your comment but still investigating why the file doesn't exist | 15:12 |
| *** saneax|AFK has quit IRC | 15:14 | |
| rfolco | mjturek, nice | 15:14 |
| *** brault has quit IRC | 15:18 | |
| *** brault has joined #oooq | 15:19 | |
| *** brault has quit IRC | 15:19 | |
| rlandy | rfolco: just fyi ... adding tests after your promotion run https://review.rdoproject.org/r/#/c/22348/ | 15:26 |
| rlandy | will check with panda about best way to execute | 15:26 |
| rfolco | rlandy, shouldn't we call the dlrn script the same way it is called in promoter service ? | 15:29 |
| rfolco | rlandy, we by pass the service, but the script inside it should be called the same way IMO | 15:29 |
| rfolco | rlandy, will comment in the patch | 15:30 |
| rlandy | rfolco: which dlrn script are you referring to? can you point out a line number so I can fix? | 15:35 |
| rfolco | rlandy, look at promotion_run step in molecule | 15:36 |
| rfolco | {{ ci_config_remote_src_dir }}/ci-scripts/dlrnapi_promoter/dlrn-promoter.sh -s | 15:36 |
| rfolco | this calls the dlrn-promoter script which is originally started by the service in production env. We don't start the service but we call the script directly as we decided in the design. | 15:37 |
| rlandy | rfolco: ack - where in my code? I call dlrn_client | 15:37 |
| rlandy | that is all I see | 15:37 |
| rfolco | rlandy, trying to understand | 15:38 |
| chandankumar | rlandy: sshnaidm please have a look at https://review.opendev.org/#/c/683126/4/zuul.d/tobiko-tripleo.yaml | 15:39 |
| chandankumar | I donot have answer there | 15:39 |
| chandankumar | rlandy: https://review.rdoproject.org/r/#/c/22535/ needs +w | 15:40 |
| rfolco | rlandy, ahh, ok I see what is missing... the same include_task we did in molecule playbook, we should call it in the zuul job workflow | 15:40 |
| rfolco | rlandy, I just don't know if we are adding these tests to molecule as well. | 15:41 |
| rfolco | I say that because we include promotion_run in molecule workflow. If you add the python tests to promotion_run.yml, its gonna run in molecule as well. | 15:42 |
| panda | rfolco: not it will not. | 15:42 |
| panda | rfolco: molecule test will have to run tests on the single componentes | 15:42 |
| rfolco | ok so lets remove that call from promotion_run and call it from somewhere else | 15:42 |
| panda | rfolco: already working on it | 15:43 |
| rfolco | rlandy, ^ | 15:43 |
| panda | rfolco: https://review.rdoproject.org/r/22538 | 15:43 |
| panda | it would have been a lot faster to just itertion on containers-push, unfortunately no part in that playbook is ready to be tested in isolation | 15:44 |
| panda | so if I want to create functional test for that, I need to refactor the whole playbook to be a role. | 15:45 |
| rlandy | chandankumar: what happens when you run a stein job with https://review.rdoproject.org/r/#/c/22535/? | 15:45 |
| chandankumar | rlandy: it have stable/stein branch created , https://github.com/openstack/tripleo-ansible/tree/stable/stein pulls older code and screwed the deployment | 15:46 |
| chandankumar | and where our tripleo-podman also does not exists | 15:46 |
| rfolco | panda, if you remove from molecule the promotion call, you remove the unit test that verifies if the dlrn-promoter is called w/ staging criteria and the api is correctly exposed | 15:47 |
| rfolco | panda, we could also have these isolated tests in molecule | 15:48 |
| sshnaidm | chandankumar, it's already included in featureset 016, why does he need fs010? https://github.com/openstack/tripleo-quickstart/blob/master/config/general_config/featureset016.yml#L20 | 15:49 |
| rfolco | panda, I agree the end-to-end test should be in zuul job... lets keep the unit tests as much as we can | 15:49 |
| rfolco | in molecule | 15:49 |
| chandankumar | sshnaidm: featureset10 comes into picture where slawq needs container-multinode jobs in CI | 15:50 |
| chandankumar | fs016 does run I think in upstream ci | 15:51 |
| panda | rfolco: agreed. Then it should be separated in another molecule test. | 15:51 |
| chandankumar | *doesnot | 15:51 |
| rfolco | panda, not default ? why? | 15:51 |
| panda | rfolco: ok in default, I don't know how your tests work, focusing on integration test ... | 15:53 |
| *** ykarel is now known as ykarel|afk | 15:54 | |
| rfolco | panda, you removed it here https://review.rdoproject.org/r/#/c/22538/4/ci-scripts/infra-setup/roles/promoter/molecule/default/playbook.yml | 15:54 |
| panda | rfolco: I didn't know there were unit tests attached to it, I'll readd it | 15:54 |
| rfolco | panda, just asking... | 15:55 |
| rfolco | panda, and added here ? https://review.rdoproject.org/r/#/c/22538/4/playbooks/staging.yml | 15:55 |
| rfolco | panda, which will run in zuul job, not molecule afaik | 15:55 |
| panda | rfolco: we can move the test to check if the dlrnapi is correctly exposed. | 15:57 |
| rfolco | panda, yeah, the dlrn-promoter script now has 2 modes, -s which calls staging.ini criteria. This is a good unit test to keep as well | 15:57 |
| *** marios has quit IRC | 15:57 | |
| * rlandy waits until panda and rfolco decide where to call this - then will complete the execution piece | 15:58 | |
| rfolco | rlandy, panda so to summarize: do not remove from molecule the promotion_run, keep it as molecule unit test and feel free to add it to zuul job workflow as a end-to-end test | 15:59 |
| panda | rfolco:ok | 16:00 |
| panda | rlandy: your test will probably be called as in https://review.rdoproject.org/r/22538 at the end of playbooks/staging.yml | 16:01 |
| *** panda is now known as panda|bbl | 16:02 | |
| rlandy | panda: ok - I can rebase on that patch and move the test | 16:02 |
| sshnaidm | chandankumar, because we switched to standalone, but if he wants - why not to run it in upstream on tobiko? | 16:02 |
| rlandy | panda: hmmmm ... that file does not have promotion run in it yet | 16:03 |
| rlandy | oh nvm | 16:04 |
| rlandy | fine | 16:04 |
| *** udesale has quit IRC | 16:10 | |
| *** dtantsur is now known as dtantsur|afk | 16:15 | |
| *** brault has joined #oooq | 16:20 | |
| weshay | arxcruz|rover, fyi.. added mean, max https://review.rdoproject.org/r/#/c/22546/ | 16:22 |
| arxcruz|rover | ack | 16:23 |
| *** holser has quit IRC | 16:24 | |
| *** brault has quit IRC | 16:27 | |
| *** brault has joined #oooq | 16:28 | |
| *** brault has quit IRC | 16:28 | |
| *** brault has joined #oooq | 16:28 | |
| *** bogdando has quit IRC | 16:28 | |
| *** jaosorior has joined #oooq | 16:29 | |
| *** brault has quit IRC | 16:29 | |
| *** brault has joined #oooq | 16:32 | |
| zbr|ruck | anyone else in PnT All Hands / CentOS meeting? | 16:38 |
| weshay | 2019-09-24T16:36:10Z E! [outputs.influxdb]: when writing to [http://influxdb:8086]: received error partial write: field type conflict: input field "container_prep_time" on measurement "build" is type float, already exists as type string dropped=6; discarding points | 16:39 |
| weshay | zbr|ruck, aye I am | 16:39 |
| weshay | zbr|ruck, so do you think infra will be on stream? | 16:40 |
| weshay | I guess that's the only choice | 16:40 |
| weshay | zbr|ruck, this is good for us.. :) | 16:40 |
| zbr|ruck | yep, probably. | 16:40 |
| weshay | finally.. centos is not downstream from RHEL | 16:40 |
| weshay | zbr|ruck, selinux too? | 16:40 |
| zbr|ruck | this should fix some of the issue we had with increasing delays caused by updates | 16:41 |
| zbr|ruck | weshay: yeah, i am sure that is one good example of issue sorted by stream | 16:43 |
| weshay | so.. I guess we'll get CentOS 9 before RHEL 9 | 16:44 |
| zbr|ruck | obviously that stream also means more risk due to being rolling, but I think overall we will be better. | 16:45 |
| zbr|ruck | not so sure about that, but we will see. | 16:45 |
| *** tesseract has quit IRC | 16:46 | |
| rfolco | rlandy, panda|bbl I said it could be any port but actually this is hardcoded to port 8787 --> https://github.com/openstack/tripleo-quickstart-extras/blob/master/roles/overcloud-prep-containers/templates/overcloud-prep-containers.sh.j2#L182 | 16:49 |
| rfolco | the target registry, the source could be any port like 5000 | 16:49 |
| chandankumar | sshnaidm: ok, no issues then | 16:50 |
| *** jaosorior has quit IRC | 16:51 | |
| *** panda|bbl is now known as panda | 16:56 | |
| panda | back | 16:56 |
| *** Vorrtex has joined #oooq | 16:56 | |
| rfolco | panda, need help, quick question: why my tripleo container image prepare does not support the same args as we use in tqe ? --output-images-file for ex | 17:10 |
| rfolco | python2-tripleoclient-12.2.1-0.20190920064104.782cd28.el7.noarch | 17:10 |
| rfolco | panda, ignore me, old arg (not master) | 17:13 |
| rfolco | https://github.com/openstack/tripleo-quickstart-extras/blob/master/roles/overcloud-prep-containers/templates/overcloud-prep-containers.sh.j2#L35 | 17:13 |
| weshay | zbr|ruck, woot.. working now http://dashboard-ci.tripleo.org/d/si1tipHZk/jobs-exploration?orgId=1&from=now-12h&to=now&fullscreen&panelId=9 | 17:22 |
| weshay | there is some old data that needs to age out.. but working! | 17:22 |
| weshay | panda, rfolco rlandy let's chat https://meet.google.com/zjw-bnub-sah?authuser=1 re: https://review.rdoproject.org/r/#/c/22445/ | 17:27 |
| *** brault has quit IRC | 17:31 | |
| *** jpena is now known as jpena|off | 17:32 | |
| arxcruz|rover | weshay: i believe container prep isn't the only issue | 17:52 |
| arxcruz|rover | the clouds also | 17:52 |
| arxcruz|rover | for example, you have two scenario001 jobs | 17:53 |
| arxcruz|rover | one 2:50 and the other 2:55 hours to finish | 17:53 |
| arxcruz|rover | one running on rax the prep container took 30 min | 17:53 |
| arxcruz|rover | the other running on ovh: 1 hour | 17:53 |
| arxcruz|rover | but just 5 minutes difference | 17:53 |
| zbr|ruck | re bld cnt: i observed huge difference on rdo: c7 would take 3x more than r8. this may provide a hint on causes. | 17:54 |
| weshay | ya.. arxcruz|rover infra is an issue.. | 17:58 |
| weshay | arxcruz|rover, if we have another suspect task.. we have a pattern to monitor it now | 17:58 |
| weshay | and compare across clouds | 17:58 |
| *** ykarel|afk is now known as ykarel|away | 18:00 | |
| *** amoralej is now known as amoralej|off | 18:02 | |
| *** dsneddon has joined #oooq | 18:05 | |
| *** Vorrtex has quit IRC | 18:07 | |
| *** Goneri has joined #oooq | 18:10 | |
| *** ykarel|away has quit IRC | 18:15 | |
| *** jtomasek has quit IRC | 18:21 | |
| *** Vorrtex has joined #oooq | 18:22 | |
| zbr|ruck | finally, I got centos8 installed via kickstart. | 18:23 |
| zbr|ruck | well, not a full success as apparently "The install command has been deprecated and no longer has any effect" | 18:24 |
| *** ykarel|away has joined #oooq | 18:35 | |
| *** d0ugal has quit IRC | 18:37 | |
| *** Goneri has quit IRC | 18:37 | |
| *** ykarel has joined #oooq | 18:42 | |
| *** ykarel|away has quit IRC | 18:43 | |
| *** d0ugal has joined #oooq | 19:02 | |
| *** ykarel has quit IRC | 19:47 | |
| rlandy | need to reboot | 19:50 |
| *** rlandy has quit IRC | 19:50 | |
| *** rlandy has joined #oooq | 19:54 | |
| *** aakarsh has joined #oooq | 20:13 | |
| *** jfrancoa has quit IRC | 20:17 | |
| *** holser has joined #oooq | 20:32 | |
| *** Vorrtex has quit IRC | 20:38 | |
| *** jschlueter has quit IRC | 21:01 | |
| *** jschlueter has joined #oooq | 21:03 | |
| *** rfolco has quit IRC | 21:11 | |
| *** rfolco has joined #oooq | 21:11 | |
| *** aakarsh has quit IRC | 21:16 | |
| *** holser has quit IRC | 21:57 | |
| *** weshay has quit IRC | 21:58 | |
| *** gchamoul has quit IRC | 22:04 | |
| *** weshay has joined #oooq | 22:19 | |
| *** rfolco has quit IRC | 22:29 | |
| *** rfolco has joined #oooq | 22:29 | |
| *** holser has joined #oooq | 22:46 | |
| *** dtantsur|afk has quit IRC | 22:51 | |
| *** aakarsh has joined #oooq | 22:55 | |
| *** rfolco has quit IRC | 23:13 | |
| *** rfolco has joined #oooq | 23:14 | |
| *** chem has quit IRC | 23:26 | |
| *** holser has quit IRC | 23:29 | |
| *** weshay has quit IRC | 23:31 | |
Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!