| *** ysandeep|away is now known as ysandeep | 03:53 | |
| *** ratailor has joined #oooq | 04:34 | |
| *** skramaja has joined #oooq | 04:49 | |
| *** ykarel has joined #oooq | 04:56 | |
| *** marios has joined #oooq | 05:32 | |
| *** udesale has joined #oooq | 05:45 | |
| *** marios has quit IRC | 06:41 | |
| *** marios has joined #oooq | 06:45 | |
| *** bogdando has joined #oooq | 06:49 | |
| *** jtomasek has joined #oooq | 06:50 | |
| *** jpena|off is now known as jpena | 07:04 | |
| *** amoralej|off is now known as amoralej | 07:09 | |
| *** holser has joined #oooq | 07:20 | |
| *** saneax has joined #oooq | 07:31 | |
| *** owalsh has quit IRC | 07:48 | |
| *** holser has quit IRC | 07:50 | |
| *** holser has joined #oooq | 07:52 | |
| *** owalsh has joined #oooq | 07:52 | |
| arxcruz|ruck | akahat|rover: good morning, how are we today? sorry, I'm not feeling well | 08:23 |
|---|---|---|
| marios | arxcruz|ruck: sorry to hear it arxcruz|ruck get well soon | 08:25 |
| *** derekh has joined #oooq | 08:31 | |
| *** apetrich has joined #oooq | 08:36 | |
| *** apetrich has quit IRC | 08:37 | |
| *** apetrich has joined #oooq | 08:38 | |
| *** jaosorior has joined #oooq | 08:44 | |
| akahat|rover | arxcruz|ruck, o/ | 08:46 |
| akahat|rover | arxcruz|ruck, we are good.. don't worry.. i | 08:46 |
| akahat|rover | i'll take care.. get well soon. | 08:46 |
| arxcruz|ruck | marios: akahat|rover wow, don't worry, I'm not sick, just not feeling well like low blood pressure | 08:48 |
| *** ykarel has quit IRC | 09:10 | |
| *** ykarel has joined #oooq | 09:11 | |
| *** ykarel is now known as ykarel|away | 09:20 | |
| *** ykarel|away has quit IRC | 09:25 | |
| ysandeep | guys, is it possible to access old taiga board tasks? | 10:01 |
| bhagyashris | marios, Hi, i am still not able to figure out why stage setup is not reaching to this point : https://github.com/rdo-infra/ci-config/blob/master/ci-scripts/dlrnapi_promoter/logic.py#L246 | 10:14 |
| bhagyashris | marios, i did stage setup on master as well and put break point there ^ but it's not reaching to that point | 10:15 |
| marios | bhagyashris: ack... i am also stepping currently with the py tests... lets discuss in more detail on our call later ? | 10:16 |
| bhagyashris | marios, ok sure | 10:17 |
| *** tosky has joined #oooq | 10:33 | |
| *** bogdando has quit IRC | 10:35 | |
| *** dtantsur|afk is now known as dtantsur | 10:38 | |
| *** jtomasek has quit IRC | 10:54 | |
| arxcruz|ruck | bhagyashris: you can use pudb to debug, it's pretty nice, and you can see graphically all the variables, I recommend | 10:57 |
| bhagyashris | arxcruz|ruck, yeah i am using pdb | 10:57 |
| *** ratailor has quit IRC | 10:57 | |
| bhagyashris | arxcruz|ruck, but issue is with code | 10:58 |
| arxcruz|ruck | bhagyashris: not pdb, pudb | 10:58 |
| bhagyashris | arxcruz|ruck, ahh ok | 10:58 |
| arxcruz|ruck | bhagyashris: https://external-content.duckduckgo.com/iu/?u=https%3A%2F%2Fpythonspot.com%2Fwp-content%2Fuploads%2F2016%2F08%2Fpython-debugging-pudb.png&f=1&nofb=1 | 10:59 |
| bhagyashris | arxcruz|ruck, ack | 11:00 |
| owalsh | hey, akahat|rover, arxcruz|ruck: is there an issue with train containers in CI? | 11:14 |
| owalsh | https://storage.bhs.cloud.ovh.net/v1/AUTH_dcaab5e32b234d56b626f72581e3644c/zuul_opendev_logs_8f1/749896/1/check/tripleo-ci-centos-8-undercloud-containers/8f10022/logs/undercloud/home/zuul/undercloud_install.log | 11:14 |
| *** jtomasek has joined #oooq | 11:14 | |
| owalsh | Not found image: https://registry-1.docker.io/v2/tripleotraincentos8/centos-binary-neutron-server/manifests/c3103a3af637b1ff751b6914c4e24f70 | 11:14 |
| *** udesale_ has joined #oooq | 11:15 | |
| *** udesale has quit IRC | 11:18 | |
| arxcruz|ruck | owalsh: let me check | 11:18 |
| owalsh | maybe just that image missing, standalone passed but it's using tripleotraincentos8/centos-binary-neutron-server-ovn | 11:19 |
| arxcruz|ruck | owalsh: I need to check with weshay|ruck we were having some issues with train on centos 8 | 11:20 |
| owalsh | arxcruz|ruck: ack, thanks | 11:23 |
| *** jtomasek has quit IRC | 11:25 | |
| ysandeep | akahat|rover, arxcruz|ruck We faced below issue on 04th in upstream multinode job, We are seeing similiar error in downstream consistently(2/2 tries). In upstream job are green now, Are you aware of any bug(including fix released) around similiar error:- | 11:26 |
| ysandeep | https://logserver.rdoproject.org/openstack-component-common/opendev.org/openstack/tripleo-ci/master/periodic-tripleo-ci-centos-8-multinode-1ctlr-featureset010-common-train/ccb13fb/logs/undercloud/home/zuul/overcloud_deploy.log.txt.gz | 11:26 |
| ysandeep | ~~~ | 11:26 |
| ysandeep | 2020-09-04 07:24:26 | Host 192.168.24.16 not found in /home/zuul/.ssh/known_hosts | 11:26 |
| ysandeep | 2020-09-04 07:24:26 | Overcloud configuration failed. | 11:26 |
| ysandeep | ~~~ | 11:26 |
| arxcruz|ruck | owalsh: this seems to happen only in train, and only one time, I'll investigate with weshay|ruck when he's available | 11:29 |
| *** jpena is now known as jpena|lunch | 11:30 | |
| arxcruz|ruck | ysandeep: Connection broken: ConnectionResetError | 11:30 |
| arxcruz|ruck | akahat|rover: have you see this happening before? | 11:31 |
| owalsh | arxcruz|ruck: https://353602d923747538603d-1a831c5ecd04a8c4708dc7397e88c9d7.ssl.cf5.rackcdn.com/750123/1/check/tripleo-ci-centos-8-containers-multinode/4a2ecbe/logs/undercloud/home/zuul/undercloud_install.log | 11:32 |
| owalsh | http://tripleo-cockpit.usersys.redhat.com/d/VTiDErfWk/jobs-exploration?orgId=1&var-influxdb_filter=branch%7C%3D%7Cstable%2Ftrain&var-influxdb_filter=job_name%7C%3D%7Ctripleo-ci-centos-8-containers-multinode | 11:34 |
| arxcruz|ruck | owalsh: oh... it seems zuul don't return the real time jobs... | 11:35 |
| arxcruz|ruck | https://zuul.openstack.org/builds?job_name=tripleo-ci-centos-8-undercloud-containers | 11:35 |
| arxcruz|ruck | I see only two | 11:35 |
| arxcruz|ruck | ah, containers multinode as well | 11:36 |
| arxcruz|ruck | I see | 11:36 |
| owalsh | yes, just centos8 containers multinode and undercloud. Standalone is ok (not using that image) | 11:37 |
| chandankumar | arxcruz|ruck: can you check whether it is happening on vexxhost cloud? | 11:45 |
| arxcruz|ruck | chandankumar: which one ? | 11:46 |
| arxcruz|ruck | so many failures | 11:46 |
| chandankumar | arxcruz|ruck: few of my failures does not have logs | 11:47 |
| arxcruz|ruck | chandankumar: i need some context | 11:47 |
| chandankumar | arxcruz|ruck: based on this message https://wiki.openstack.org/wiki/Infrastructure_Status -> deleted bogus 2001:db8:: addresses on mirror01.ca-ymq-1.vexxhost.opendev.org once more does the conection broken error is happing due to vexxhost cloud | 11:48 |
| *** jtomasek has joined #oooq | 11:50 | |
| arxcruz|ruck | chandankumar: okay, so, what exactly is happening, which jobs are you seeing failing ? | 11:50 |
| chandankumar | for my case: logs are not there, https://storage.gra.cloud.ovh.net/v1/AUTH_dcaab5e32b234d56b626f72581e3644c/zuul_opendev_logs_7eb/750119/4/check/tripleo-ci-centos-8-scenario002-standalone/7ebdca4/logs/ -> undercloud logs are not there | 11:51 |
| chandankumar | and https://2cab3e164952e2b59e5f-9f3c64f3d814a9e4d0be6167f83e8b00.ssl.cf2.rackcdn.com/749645/22/check/tripleo-ci-centos-8-content-provider/7362598/ logs are also not there but the job failed | 11:52 |
| *** jtomasek has quit IRC | 12:00 | |
| akahat|rover | arxcruz|ruck, ysandeep hey.. | 12:06 |
| akahat|rover | no i didn't seen those error before.. | 12:06 |
| akahat|rover | ysandeep, have you filed bug for it? | 12:06 |
| ysandeep | akahat|rover, no | 12:07 |
| arxcruz|ruck | akahat|rover: would you mind to fill the bug? | 12:09 |
| marios | panda | 12:10 |
| akahat|rover | arxcruz|ruck, yes.. i'll... i'm looking in to it.. | 12:11 |
| arxcruz|ruck | ok | 12:11 |
| arxcruz|ruck | ty | 12:11 |
| *** ysandeep is now known as ysandeep|afk | 12:15 | |
| *** jpena|lunch is now known as jpena | 12:30 | |
| akahat|rover | ysandeep|afk, arxcruz|ruck i've looked in to the jobs.. most of the jobs what i found are failing with the tripleotraincentos8/centos-binary-neutron-server image not found. | 12:35 |
| akahat|rover | arxcruz|ruck, ysandeep|afk i'm not quite sure about the sskey issue.. so filing bug at the moment is not good idea.. if job fails with the same error again.. then we can.. | 12:36 |
| akahat|rover | ssh key issue * | 12:36 |
| arxcruz|ruck | akahat|rover: add a note in our ruck rover at least | 12:37 |
| akahat|rover | arxcruz|ruck, yup.. | 12:37 |
| akahat|rover | arxcruz|ruck, https://bugs.launchpad.net/tripleo/+bug/1894664 image is not present in the docker registry.. | 12:41 |
| openstack | Launchpad bug 1894664 in tripleo "[train] containers and containers-multinode jobs failing with container not found" [Critical,Triaged] | 12:41 |
| arxcruz|ruck | akahat|rover: ?? | 12:42 |
| *** ysandeep|afk is now known as ysandeep | 12:42 | |
| akahat|rover | arxcruz|ruck, tripleotraincentos8/centos-binary-neutron-server this image is not present in the docker hub. | 12:43 |
| arxcruz|ruck | akahat|rover: yup, that's why is failing | 12:43 |
| ysandeep | akahat|rover, ack, thanks! | 12:44 |
| akahat|rover | arxcruz|ruck, can you point me how to fix it? | 12:44 |
| akahat|rover | marios, chandankumar ^^ | 12:44 |
| arxcruz|ruck | akahat|rover: I'm not sure tbh | 12:44 |
| marios | akahat|rover: in call will check in bit? | 12:53 |
| akahat|rover | marios, ok | 13:01 |
| *** jtomasek has joined #oooq | 13:01 | |
| marios | tripleo-ci scrum | 13:01 |
| chandankumar | akahat|rover: how it got missed? | 13:02 |
| chandankumar | akahat|rover: you need to manually pull and push the container to docker.io from promoter | 13:07 |
| akahat|rover | chandankumar, ah.. that's weird. :| | 13:08 |
| bhagyashris | marios, rebased and pushed the patch | 13:28 |
| marios | bhagyashris: thank you! | 13:42 |
| *** jtomasek has quit IRC | 13:50 | |
| arxcruz|ruck | marios: akahat|rover chandankumar it's related to train, i know weshay|ruck did some manua changes on train last friday, I'm not sure if manually push the container without his approval would be a good idea | 13:54 |
| chandankumar | arxcruz|ruck: gates are busted na? | 13:55 |
| arxcruz|ruck | chandankumar: only stable/train | 13:56 |
| arxcruz|ruck | chandankumar: do you know how to manually push the container? | 13:57 |
| chandankumar | arxcruz|ruck: I have not done that, marios might knows | 13:59 |
| arxcruz|ruck | marios: ^ | 14:00 |
| marios | arxcruz|ruck: chandankumar: e.g. https://gist.github.com/weshayutin/692c80f2d4179dcf2108265bd9218bc9 | 14:01 |
| marios | buildah push 02a65dcb3a4f docker.io/tripleomaster/centos-binary-horizon:0481ffa881dbc1aa61d9810533d85229 | 14:01 |
| arxcruz|ruck | man, i'm really not comfortable doing that | 14:01 |
| chandankumar | arxcruz|ruck: only one container is missing na? | 14:05 |
| *** jbadiapa has joined #oooq | 14:06 | |
| chandankumar | arxcruz|ruck: I have pulled the missing container locallu | 14:09 |
| chandankumar | arxcruz|ruck: https://bugs.launchpad.net/tripleo/+bug/1894664 | 14:09 |
| openstack | Launchpad bug 1894664 in tripleo "[train] containers and containers-multinode jobs failing with container not found" [Critical,Triaged] | 14:09 |
| arxcruz|ruck | chandankumar: you did? | 14:10 |
| chandankumar | arxcruz|ruck: I have not pushed yet | 14:10 |
| arxcruz|ruck | let me add you in the email i sent to weshay|ruck | 14:10 |
| chandankumar | let me push that one | 14:12 |
| *** jpena is now known as jpena|away | 14:12 | |
| chandankumar | arxcruz|ruck: I have pushed the images | 14:17 |
| chandankumar | *image | 14:17 |
| arxcruz|ruck | chandankumar: dhanyavaad | 14:18 |
| arxcruz|ruck | owalsh: it seems chandankumar pushed the missing image, may you recheck your patch ? | 14:18 |
| *** udesale_ has quit IRC | 14:27 | |
| * chandankumar out, see ya tomorrow | 14:31 | |
| *** saneax has quit IRC | 15:12 | |
| *** jpena|away is now known as jpena | 15:47 | |
| owalsh | arxcruz|ruck: ack, was afk for a bit. Should have result for https://review.opendev.org/750123 before my patch | 15:51 |
| *** marios has quit IRC | 15:58 | |
| *** jpena is now known as jpena|away | 16:15 | |
| *** ysandeep is now known as ysandeep|away | 16:40 | |
| *** amoralej is now known as amoralej|off | 16:45 | |
| *** jpena|away is now known as jpena | 16:54 | |
| *** derekh has quit IRC | 17:03 | |
| *** dtantsur is now known as dtantsur|afk | 17:18 | |
| *** jbadiapa has quit IRC | 17:22 | |
| *** jpena is now known as jpena|off | 17:23 | |
| *** skramaja has quit IRC | 18:11 | |
| *** apetrich has quit IRC | 19:36 | |
| *** apetrich has joined #oooq | 21:53 | |
| *** apetrich has quit IRC | 21:53 | |
| *** tosky has quit IRC | 23:16 | |
Generated by irclog2html.py 2.17.2 by Marius Gedminas - find it at https://mg.pov.lt/irclog2html/!
