| *** tosky has quit IRC | 00:05 | |
| *** holser_ has quit IRC | 00:14 | |
| *** weshay has quit IRC | 00:19 | |
| *** weshay has joined #oooq | 00:20 | |
| hubbot1 | FAILING CHECK JOBS on master: tripleo-ci-centos-7-standalone-upgrade, tripleo-ci-centos-7-scenario012-standalone, tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset053, tripleo-ci-centos-7-scenario008-multinode-oooq-container, tripleo-ci-centos-7-scenario010-multinode-oooq-container @ https://review.openstack.org/604298, stable/pike: tripleo-ci-centos-7-ovb-3ctlr_1comp_1supp-featureset039 @ (2 more messages) | 00:28 |
|---|---|---|
| *** hamzy has joined #oooq | 00:44 | |
| *** yolanda has quit IRC | 01:07 | |
| *** yolanda has joined #oooq | 01:15 | |
| *** rlandy has quit IRC | 01:55 | |
| hubbot1 | FAILING CHECK JOBS on master: tripleo-ci-centos-7-standalone-upgrade, tripleo-ci-centos-7-scenario012-standalone, tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset053, tripleo-ci-centos-7-scenario008-multinode-oooq-container, tripleo-ci-centos-7-scenario010-multinode-oooq-container @ https://review.openstack.org/604298, stable/pike: tripleo-ci-centos-7-ovb-3ctlr_1comp_1supp-featureset039 @ (2 more messages) | 02:28 |
| *** ykarel has joined #oooq | 03:22 | |
| *** weshay has quit IRC | 03:24 | |
| *** weshay has joined #oooq | 03:24 | |
| *** weshay has quit IRC | 03:24 | |
| *** weshay has joined #oooq | 03:25 | |
| *** raukadah is now known as chandankumar | 03:53 | |
| *** udesale has joined #oooq | 03:56 | |
| hubbot1 | FAILING CHECK JOBS on master: tripleo-ci-centos-7-standalone-upgrade, tripleo-ci-centos-7-scenario012-standalone, tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset053, tripleo-ci-centos-7-scenario008-multinode-oooq-container, tripleo-ci-centos-7-scenario010-multinode-oooq-container @ https://review.openstack.org/604298, stable/pike: tripleo-ci-centos-7-ovb-3ctlr_1comp_1supp-featureset039 @ (2 more messages) | 04:28 |
| *** ykarel has quit IRC | 04:47 | |
| *** ykarel has joined #oooq | 05:02 | |
| *** ratailor has joined #oooq | 05:34 | |
| *** holser_ has joined #oooq | 05:39 | |
| *** holser_ has quit IRC | 05:54 | |
| *** skramaja has joined #oooq | 06:13 | |
| *** skramaja_ has joined #oooq | 06:21 | |
| *** skramaja has quit IRC | 06:22 | |
| hubbot1 | FAILING CHECK JOBS on master: tripleo-ci-centos-7-standalone-upgrade, tripleo-ci-centos-7-scenario012-standalone, tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset053, tripleo-ci-centos-7-scenario008-multinode-oooq-container, tripleo-ci-centos-7-scenario010-multinode-oooq-container @ https://review.openstack.org/604298, stable/pike: tripleo-ci-centos-7-ovb-3ctlr_1comp_1supp-featureset039 @ (2 more messages) | 06:28 |
| Tengu | hello there! | 06:32 |
| quiquell|off | Tengu: o/ | 06:33 |
| Tengu | can anyone help me on https://bugs.launchpad.net/tripleo/+bug/1822120 ? I'm trying to run the reproducer, but it fails always one "Wait for zuul tenant" while running on my builder :( | 06:33 |
| openstack | Launchpad bug 1822120 in tripleo "[CI][Queens] undercloud-upgrade task fails due to rabbitmq" [Critical,Triaged] - Assigned to Cédric Jeanneret (cjeanner) | 06:33 |
| Tengu | quiquell|off: «o/ | 06:33 |
| Tengu | so I'm basically unable to have a reproducer in order to understand what's going on with that bloody rabbitmq. and it's a bit annoying, since it's a blocker on queens branch. | 06:33 |
| quiquell|off | Tengu: it takes some times to come up the zuul tenant the first t ime | 06:34 |
| Tengu | quiquell|off: I've waited until the timeout.... | 06:34 |
| Tengu | apparently there's an issue with resolution within the tripleo-ci-reproducer_web_1 container | 06:35 |
| quiquell|off | Tengu: do you have the docker-compose logs ? | 06:35 |
| Tengu | where is it supposed to be located? | 06:35 |
| quiquell|off | Tengu: first of all this reproducer have some restrictions on the gerrit ssh keys | 06:36 |
| Tengu | hmmm I'm not even at that point... | 06:36 |
| quiquell|off | Tengu: Is in the READEM of the reproducer | 06:36 |
| quiquell|off | Tengu: the ssh key part is a little pain :-/ | 06:37 |
| Tengu | yeah, I've seen it wants my private key in order to access gerrit. | 06:37 |
| *** quiquell|off is now known as quiquell | 06:37 | |
| quiquell | Tengu: you can create new ones with the constraints and pass them to the reproducer script | 06:37 |
| quiquell | Tengu: they keys have to be passwordless and should have -----BEGIN RSA PRIVATE KEY----- -----END RSA PRIVATE KEY----- | 06:38 |
| quiquell | Tengu: if this is the case you have to do nothing | 06:38 |
| quiquell | Tengu: if not you have to create new one and add it to gerrit | 06:39 |
| *** udesale has quit IRC | 06:41 | |
| *** udesale has joined #oooq | 06:41 | |
| Tengu | yeah. have to wait for the new one to be deployed. | 06:41 |
| Tengu | still - the issue doesn't involve ssh connection so far. | 06:41 |
| Tengu | http://paste.openstack.org/show/748562/ | 06:42 |
| Tengu | looks like it's missing some entry in the /etc/hosts file in the container | 06:42 |
| Tengu | or something like that. | 06:42 |
| Tengu | thus it returns a 500 error, thus timeout of the playbook | 06:42 |
| Tengu | anyway - lemme re-try... | 06:44 |
| quiquell | Wait | 06:44 |
| quiquell | Go to ~/tripleo-ci-reproducer | 06:45 |
| quiquell | docker-compose ps | 06:45 |
| Tengu | ah, logs are supposed to be there ? | 06:45 |
| Tengu | nah, already cleaned the env. I'll check in a couple of minutes, it's running again. | 06:45 |
| quiquell | To see what services are not ok | 06:45 |
| quiquell | ack | 06:46 |
| quiquell | you have a docker-compose zuul at that directory | 06:46 |
| quiquell | with all the services it needs | 06:47 |
| Tengu | yep, just saw that. | 06:48 |
| Tengu | and an empty "logs" directory. | 06:48 |
| Tengu | at least empty for now. | 06:48 |
| quiquell | Log directory is the fake log server place | 06:52 |
| quiquell | So just job output logs | 06:52 |
| quiquell | Not docker compose logs | 06:52 |
| Tengu | ok | 06:52 |
| quiquell | For docker compose logs | 06:52 |
| quiquell | Just run docker-compose logs | 06:53 |
| quiquell | The main zuul service is scheduler | 06:53 |
| ykarel | Tengu, quiquell i think the issue is access to gerrit stream, Tengu do you have access to gerrit stream? | 06:54 |
| Tengu | ykarel: define "gerrit stream" ? | 06:55 |
| ykarel | Tengu, ssh review.rdoproject.org -p 29418 -l <username> gerrit stream-events | 06:55 |
| ykarel | replace username with your user | 06:55 |
| ykarel | rdo gerrit user | 06:55 |
| Tengu | hmmm "not happy". | 06:55 |
| Tengu | permission denied. | 06:55 |
| ykarel | you using wrong keys | 06:56 |
| Tengu | ah, for the new key. | 06:56 |
| Tengu | the "old" one is OK apparently. | 06:56 |
| Tengu | "Capability streamEvents is required to access this resource" | 06:56 |
| ykarel | yes you don't have access ^^ | 06:56 |
| Tengu | how convenient....... how are we supposed to use the reproducer then? pfff | 06:56 |
| ykarel | :) | 06:56 |
| Tengu | old one was old, but at least it did work. apparently not anymore thought. | 06:57 |
| Tengu | *though | 06:57 |
| Tengu | tried it, wants openstack + heat commands ON THE HOST | 06:57 |
| ykarel | old one still works i think for some cases | 06:57 |
| quiquell | ykarel: always forget about it yep the gerrit stream it is | 06:57 |
| Tengu | not for queens reproducer, unless you install openstack. #lolz. | 06:57 |
| ykarel | i hacked new one for my use atleast | 06:57 |
| ykarel | i setup new sf instance and mocked review.rdoproject.org to use it | 06:58 |
| quiquell | Tengu new one is still little alpha | 06:58 |
| Tengu | that said.... | 06:59 |
| Tengu | 2019-03-29 07:01:40,645 - gear.Client.b'unknown' - DEBUG - Unable to connect to <gear.Connection 0x7f38239bff28 host: scheduler port: 4730> | 06:59 |
| Tengu | that's the scheduler port, not the gerrit stream | 06:59 |
| Tengu | I'm not even at gerrit fetch/whatever step. | 06:59 |
| ykarel | Tengu, yes right, see scheduler logs, you will find issue is gerrit stream | 06:59 |
| Tengu | it fails before that.... | 06:59 |
| ykarel | scheduler must be down | 06:59 |
| Tengu | paramiko.ssh_exception.SSHException: not a valid OPENSSH private key file | 07:00 |
| Tengu | what the actual fuck ? | 07:00 |
| quiquell | ykarel: can you give tengo access to stream? | 07:00 |
| Tengu | the key is a PEM, password-less | 07:00 |
| Tengu | 2019-03-29 07:02:23,586 - paramiko.transport - INFO - Authentication (publickey) failed. | 07:00 |
| ykarel | Tengu, u created keys as described in workaround? | 07:00 |
| Tengu | hmmmm. | 07:00 |
| Tengu | ykarel: yep. | 07:00 |
| ykarel | and added those keys to both gerrits? | 07:01 |
| Tengu | ssh-keygen -t rsa -m PEM -b 4096 - without password. | 07:01 |
| Tengu | yep | 07:01 |
| Tengu | I have two keys now. | 07:01 |
| quiquell | Then it should be ok :-/ | 07:01 |
| Tengu | BUT.... apparently paramiko returns that invalid key due to the 2019-03-29 07:02:23,586 - paramiko.transport - INFO - Authentication (publickey) failed. | 07:01 |
| Tengu | so either my key isn't yet deployed, or there's something else somewhere..... | 07:01 |
| quiquell | Tengu: can you try ssh into upstream/rdo gerrits with those keys ? | 07:02 |
| quiquell | Just make sure they work | 07:02 |
| Tengu | I'm using a "stack" user on my builder.... maybe it wants to use that $USER ? | 07:02 |
| Tengu | ssh review.rdoproject.org -p 29418 -l cjeanner gerrit stream-events // Permission denied (publickey). | 07:02 |
| Tengu | so yeah, new key is probably not deployed at some point. | 07:02 |
| quiquell | ykarel: ^ can you give Tengu permissions there ? | 07:03 |
| ykarel | quiquell, there is a patch to open for all registered user:- https://review.rdoproject.org/r/#/c/19352/ | 07:03 |
| Tengu | ... and I don't really want to remove the password on the master ssh key. | 07:03 |
| quiquell | ykarel: any stopper on merge that ? | 07:04 |
| ykarel | quiquell, don't know i commented and waiting | 07:04 |
| quiquell | ykarel: let's ask at infra | 07:04 |
| ykarel | quiquell, you can propose patch to add tengu | 07:04 |
| ykarel | meanwhile | 07:04 |
| ykarel | add me as well so i don't need to use hack | 07:04 |
| quiquell | ykarel: let me find the place | 07:05 |
| ykarel | quiquell, https://review.rdoproject.org/r/#/q/file:%2509resources/gerrit-stream.yaml | 07:05 |
| quiquell | ykarel: you are indexman :-) | 07:05 |
| ykarel | it's just gerrit search :) | 07:07 |
| quiquell | Tengu: what's your RDO email ? | 07:07 |
| quiquell | ahh redhat one | 07:07 |
| Tengu | yep | 07:07 |
| quiquell | cjeanner@redhat.com | 07:07 |
| quiquell | ^ ? | 07:07 |
| ykarel | quiquell, add my also: ykarel@redhat.com | 07:07 |
| quiquell | ykarel: already eon | 07:07 |
| *** kopecmartin|off is now known as kopecmartin | 07:07 | |
| Tengu | brb, lunch in the meantime. | 07:09 |
| marios_|ruck | o/ folks | 07:18 |
| quiquell | marios_|ruck: hello there | 07:19 |
| *** quiquell is now known as quiquell|brb | 07:25 | |
| Tengu | so I should have the stream access now, tested and working. | 07:25 |
| ykarel | cool | 07:27 |
| Tengu | now time to re-re-re-retry the reproducer :). | 07:29 |
| chandankumar | quiquell|brb: marios_|ruck https://review.openstack.org/#/c/648373/ please have a look when free! | 07:29 |
| marios_|ruck | chandankumar: ack | 07:29 |
| Tengu | ah. right. it's borked with the virtualenv. mandatory to remove all ~/.local/* content... | 07:30 |
| *** apetrich has joined #oooq | 07:30 | |
| *** jaosorior has quit IRC | 07:37 | |
| *** ykarel is now known as ykarel|lunch | 07:42 | |
| ykarel|lunch | Tengu, is rabbitmq issue started recently? | 07:42 |
| ykarel|lunch | queens just saw the lp | 07:43 |
| Tengu | ykarel|lunch: no idea, I didn't submit anything to queens for a while | 07:43 |
| ykarel|lunch | Tengu, looking https://bugs.launchpad.net/tripleo/+bug/1822120 | 07:43 |
| openstack | Launchpad bug 1822120 in tripleo "[CI][Queens] undercloud-upgrade task fails due to rabbitmq" [Critical,Triaged] - Assigned to Cédric Jeanneret (cjeanner) | 07:43 |
| Tengu | just stumbled on that with my puppet-tripleo change | 07:43 |
| Tengu | yeh | 07:43 |
| Tengu | petty one :(. | 07:43 |
| ykarel|lunch | this issue started recently? | 07:43 |
| Tengu | as said: no idea if it was already present before. | 07:44 |
| ykarel|lunch | okk, if it's started after 27th, we updated rabbitmq recently | 07:44 |
| ykarel|lunch | in all releases | 07:44 |
| ykarel|lunch | so u can related with that | 07:45 |
| ykarel|lunch | https://review.rdoproject.org/r/#/c/15234/ | 07:45 |
| Tengu | well, that's pretts sure the root cause. | 07:45 |
| Tengu | and as nobody tested anything as usual.... | 07:46 |
| * Tengu flips tables | 07:46 | |
| ykarel|lunch | Tengu, job history:- http://zuul.openstack.org/builds?job_name=tripleo-ci-centos-7-undercloud-upgrades&branch=stable%2Fqueens | 07:46 |
| ykarel|lunch | so looks like started after 27th so seems related | 07:46 |
| ykarel|lunch | but good to reproduce and have a fix | 07:47 |
| Tengu | ykarel|lunch: I've hooked dciabrin from PIDONE | 07:47 |
| ykarel|lunch | yeah he can fix | 07:47 |
| Tengu | yup. he's conducting some checks on his own. stay tuned - need to fill my tea pot | 07:48 |
| ykarel|lunch | okk, /me too going for lunch | 07:49 |
| ykarel|lunch | i just checked failing jobs have new rabbitmq | 07:49 |
| quiquell|brb | Tengu: you have gerrit stream permissions at RDO now, try again | 07:57 |
| Tengu | quiquell|brb: yep, already tests, but apparently theres something else in my setup. | 07:57 |
| quiquell|brb | damn... ok | 07:57 |
| Tengu | probably the fact I'm using "stack" user on my server.... | 07:57 |
| Tengu | anyway. we have the root cause apparently. | 07:57 |
| quiquell|brb | Tengu: you can override that too | 07:58 |
| quiquell|brb | Tengu: ack | 07:58 |
| Tengu | upgrading rabbitmq-server from 3.6.5 to 3.6.15 for queens, without testing, was probably a bad idea. | 07:58 |
| quiquell|brb | Tengu: if you want we can tmate when I am fully operational again | 07:58 |
| quiquell|brb | Tengu: hehe even if it passes is a bad idea :-) | 07:58 |
| Tengu | quiquell|brb: yeah. so Damien is a bit... well. "not really happy with that" :). He takes over the issue. | 07:59 |
| Tengu | quiquell|brb: ah, -ug rg apparently. | 08:02 |
| Tengu | should be OK if I set both to "cjeanner". | 08:02 |
| *** quiquell|brb is now known as quiquell | 08:06 | |
| *** ccamacho has joined #oooq | 08:07 | |
| Tengu | does anyone know wich rabbitmq module is used? | 08:10 |
| *** amoralej|off is now known as amoralej | 08:14 | |
| *** tosky has joined #oooq | 08:15 | |
| marios_|ruck | rfolco: ah sorry didn't gt to it on time :/ https://review.rdoproject.org/r/#/q/status:open+project:config+branch:master+topic:oooq/branching | 08:16 |
| * marios_|ruck was looking forward | 08:16 | |
| quiquell | marios_|ruck: Have merge all of them | 08:18 |
| Tengu | quiquell: "ansible-role-tripleo-ci-reproducer : Wait for job to start" \o/ | 08:22 |
| Tengu | so now.... well, have to check what's going on at that step :) | 08:22 |
| Tengu | bah. timed out. | 08:23 |
| quiquell | Tengu: not all is lost | 08:24 |
| quiquell | Tengu: could be that the job is taking more than expected | 08:24 |
| quiquell | just open zuul web | 08:24 |
| quiquell | localhost:9090 | 08:24 |
| Tengu | damn, doesn't work with curl -.- | 08:25 |
| Tengu | quiquell: queue: 0 | 08:26 |
| Tengu | guess it's bad. | 08:26 |
| quiquell | not goo dno | 08:26 |
| quiquell | click go builds | 08:26 |
| quiquell | see if there is any | 08:26 |
| Tengu | empty | 08:26 |
| marios_|ruck | Tengu: omg where | 08:26 |
| * marios_|ruck checks grafana | 08:26 | |
| Tengu | marios_|ruck: locally | 08:26 |
| Tengu | marios_|ruck: trying the reproducer on my own server | 08:27 |
| marios_|ruck | Tengu: oh cool you're playing with reproducer | 08:27 |
| marios_|ruck | nice | 08:27 |
| Tengu | *trying to | 08:27 |
| Tengu | and losing my mind ;) | 08:27 |
| * marios_|ruck waiting for things to fail and jumpy sorry | 08:27 | |
| marios_|ruck | i mean | 08:27 |
| marios_|ruck | its friday after all | 08:27 |
| Tengu | but it's Friday | 08:27 |
| Tengu | :D | 08:27 |
| marios_|ruck | what could possibly go wrong | 08:27 |
| *** dsneddon has joined #oooq | 08:27 | |
| Tengu | no-thing obviously ;) | 08:27 |
| hubbot1 | FAILING CHECK JOBS on master: tripleo-ci-centos-7-standalone-upgrade, tripleo-ci-centos-7-scenario012-standalone, tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset053, tripleo-ci-centos-7-scenario008-multinode-oooq-container, tripleo-ci-centos-7-scenario010-multinode-oooq-container @ https://review.openstack.org/604298, stable/pike: tripleo-ci-centos-7-ovb-3ctlr_1comp_1supp-featureset039 @ (2 more messages) | 08:28 |
| Tengu | who's in for some tmate session in order to understand what's going on ? :) | 08:29 |
| quiquell | Let's do one | 08:29 |
| quiquell | have time now | 08:29 |
| *** dsneddon has quit IRC | 08:32 | |
| *** fmount has quit IRC | 08:33 | |
| *** fmount has joined #oooq | 08:37 | |
| *** jpena|off is now known as jpena | 08:40 | |
| *** ykarel|lunch is now known as ykarel | 08:49 | |
| *** udesale has quit IRC | 08:57 | |
| *** udesale has joined #oooq | 08:58 | |
| *** dsneddon has joined #oooq | 09:02 | |
| *** udesale has quit IRC | 09:07 | |
| *** bogdando has joined #oooq | 09:07 | |
| *** dsneddon has quit IRC | 09:08 | |
| *** udesale has joined #oooq | 09:10 | |
| *** udesale has quit IRC | 09:15 | |
| *** dsneddon has joined #oooq | 09:28 | |
| *** dsneddon has quit IRC | 09:33 | |
| *** fmount has left #oooq | 09:41 | |
| *** dsneddon has joined #oooq | 09:42 | |
| *** dtantsur|afk is now known as dtantsur | 09:43 | |
| quiquell | marios_|ruck, ykarel: we need this to tag tripleo-ci-testing at fedora https://review.rdoproject.org/r/#/c/19842/ | 09:45 |
| quiquell | to be able to build containers at fedora for stein | 09:46 |
| *** dsneddon has quit IRC | 09:47 | |
| zbr | quiquell: re stein, do you think we also need https://review.rdoproject.org/r/#/c/19754/ ? | 09:48 |
| quiquell | zbr: If we want to exercise at check | 09:50 |
| quiquell | zbr: at RDO | 09:50 |
| quiquell | zbr: I think we have same upstream | 09:50 |
| quiquell | zbr: is good to have though | 09:50 |
| zbr | quiquell: that was the idea, sooner of later they will diverge and having this job could save us time | 09:50 |
| quiquell | the review is missing some stuff | 09:51 |
| zbr | i also have https://review.rdoproject.org/r/#/c/19754/ which is a side-effect of my image work, that i will like to get in. it will be useful in 6mo | 09:51 |
| quiquell | let me add a pointer on it | 09:51 |
| quiquell | so you replicate | 09:51 |
| zbr | sure | 09:51 |
| quiquell | zbr: commented https://review.rdoproject.org/r/#/c/19754/ | 09:53 |
| ykarel | quiquell, ack +2 | 09:53 |
| quiquell | ok merging | 09:54 |
| quiquell | let's rerun containers build | 09:54 |
| ykarel | quiquell, i saw fedora build containers failed in last run | 09:54 |
| ykarel | seems real issue: tripleo_repos.main.InvalidArguments: Branches only suppported with centos7 | 09:54 |
| quiquell | yep need this as consistent-to-tripleo-ci-testing is not running for it | 09:54 |
| quiquell | I can manually promote DLRN though | 09:54 |
| quiquell | and just run the job | 09:54 |
| ykarel | mmm but can't get how that job is related with ^^ ERROR | 09:54 |
| ykarel | okk repo not exist | 09:55 |
| quiquell | I think a Depends-On is wrong | 09:55 |
| quiquell | Yep let me fix Depends-On | 09:56 |
| quiquell | done | 09:56 |
| quiquell | Let's see now | 09:56 |
| ykarel | it even failed before:- tripleo-repos -b stein --mirror https://mirrors.fedoraproject.org --rdo-mirror https://trunk.rdoproject.org tripleo-ci-testing | 09:57 |
| ykarel | -b stein not supported | 09:57 |
| ykarel | http://git.openstack.org/cgit/openstack/tripleo-repos/tree/tripleo_repos/main.py#n230 | 09:57 |
| quiquell | It needs https://review.rdoproject.org/r/19335 | 09:57 |
| quiquell | but the Depends-On was missing the "r" | 09:57 |
| ykarel | okk found https://review.openstack.org/#/c/648485/1/tripleo_repos/main.py | 09:58 |
| quiquell | yep | 09:58 |
| ykarel | quiquell, but don't you guys running fedora job in master? | 09:58 |
| quiquell | Ahh shit | 09:58 |
| ykarel | that patch seems harding just to stein | 09:59 |
| quiquell | ... | 09:59 |
| quiquell | wait the important one is the upstream one the RDO does not matter | 09:59 |
| quiquell | I have mix them | 09:59 |
| *** dsneddon has joined #oooq | 10:00 | |
| quiquell | ykarel: nah forget about it let me debug | 10:00 |
| quiquell | but issue is fedora not being promoted to tripleo-ci-testing | 10:01 |
| quiquell | because we don't call the fedora promoting logic | 10:01 |
| quiquell | we do release: stein in that build container job so let's see now | 10:02 |
| quiquell | ok now is promoted | 10:02 |
| quiquell | with the config fix | 10:02 |
| ykarel | anyway that patch is hardening stein, so will not work for other branches | 10:05 |
| quiquell | ykarel: witch one ? | 10:05 |
| *** dsneddon has quit IRC | 10:05 | |
| ykarel | https://review.openstack.org/#/c/648485/1/tripleo_repos/main.py | 10:05 |
| ykarel | okk sorry, should work i think | 10:06 |
| ykarel | looked again | 10:06 |
| ykarel | it has a top condition on args.branch | 10:06 |
| ykarel | so should work with both master and stable/stein | 10:07 |
| quiquell | well still something is missing | 10:08 |
| quiquell | it's going to work but | 10:08 |
| quiquell | we need and else with "not in centos7" | 10:08 |
| quiquell | Run image build \o/ yey! | 10:10 |
| quiquell | marios_|ruck, zbr, ykarel: to do manual pushing to docker hubg of stein fedora images https://review.rdoproject.org/r/#/c/19335/ | 10:20 |
| zbr | quiquell: commented. | 10:24 |
| zbr | quiquell: and a sidenote: not trying to enforce my terminology, but if we do it, lets agree on one and stick to it everywhere (is still easy to change the release config code) | 10:26 |
| zbr | before reading your change i was wondering if it would not be easier to use release_flavor = distro_name + distro_version | 10:27 |
| hubbot1 | FAILING CHECK JOBS on master: tripleo-ci-centos-7-standalone-upgrade, tripleo-ci-centos-7-scenario012-standalone, tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset053, tripleo-ci-centos-7-scenario008-multinode-oooq-container, tripleo-ci-centos-7-scenario010-multinode-oooq-container @ https://review.openstack.org/604298, stable/pike: tripleo-ci-centos-7-ovb-3ctlr_1comp_1supp-featureset039 @ (2 more messages) | 10:28 |
| quiquell | zbr: release files has nothing to do with this | 10:34 |
| quiquell | zbr: the followup is this https://review.rdoproject.org/r/19846 | 10:34 |
| zbr | quiquell: is about terminology, we should use the same everywhere to avoid confusions. | 10:34 |
| zbr | sadly I see nobody proposing a good name for the os-version combined tuple | 10:36 |
| zbr | ... maybe i am bit ocd | 10:37 |
| *** dsneddon has joined #oooq | 10:40 | |
| zbr | marios_|ruck: hi! are you here? | 10:49 |
| marios_|ruck | zbr: sup | 10:52 |
| zbr | marios_|ruck: quiquell : https://etherpad.openstack.org/p/tripleo-variables lets find some names to agree on | 10:54 |
| *** jaosorior has joined #oooq | 10:55 | |
| marios_|ruck | zbr: ack me cherrypick hell will have think about it in bit | 11:03 |
| zbr | sure. | 11:04 |
| *** jaosorior has quit IRC | 11:09 | |
| *** weshay is now known as weshay|rover | 12:06 | |
| weshay|rover | marios_|ruck morning, things are looking pretty good :) need anything? | 12:06 |
| marios_|ruck | weshay|rover: o/ | 12:08 |
| marios_|ruck | weshay|rover: yeah nothing new afaics... | 12:08 |
| marios_|ruck | weshay|rover: queens is the only blocked thing on https://bugs.launchpad.net/tripleo/+bug/1822080 | 12:08 |
| openstack | Launchpad bug 1822080 in tripleo "periodic centos-7-ovb-1ctlr_1comp-featureset020-queens failing tempest "Endpoint not found"" [Critical,Triaged] - Assigned to Martin Kopec (mkopec) | 12:08 |
| weshay|rover | k | 12:09 |
| marios_|ruck | weshay|rover: spent some time in cherrypick hell and posted some puppet-cinder but need to chat with abishop | 12:09 |
| marios_|ruck | weshay|rover: master rocky pike promoted again today | 12:10 |
| weshay|rover | hellz ya | 12:10 |
| marios_|ruck | weshay|rover: and phase 1 rocky | 12:10 |
| weshay|rover | ya.. the promotions section of cockpit is all lit up | 12:11 |
| marios_|ruck | weshay|rover: as usual known stuff in https://etherpad.openstack.org/p/ruckroversprint7 | 12:12 |
| weshay|rover | config core... https://review.rdoproject.org/r/#/c/19812/ | 12:13 |
| weshay|rover | marios_|ruck https://review.openstack.org/#/c/648386/ if you have a sec | 12:19 |
| *** trown|outtypewww is now known as trown | 12:20 | |
| weshay|rover | marios_|ruck ah.. stein is in the pipeline :) | 12:20 |
| marios_|ruck | weshay|rover: i believe quique was working there earlier | 12:21 |
| *** ratailor has quit IRC | 12:21 | |
| marios_|ruck | weshay|rover: looking | 12:22 |
| weshay|rover | 2019-03-29 05:43:252019-03-29 05:37:50openstack-tripleo-heat-templates6635dc6329504baddb9b594a0eb1505aa805a740SUCCESSrepobuild log | 12:22 |
| weshay|rover | yup and it's pulling new hashes | 12:22 |
| weshay|rover | nice | 12:22 |
| chandankumar | arxcruz: quiquell please have a look at these two patches https://review.openstack.org/#/c/648373/ and https://review.openstack.org/#/c/648617/ | 12:22 |
| chandankumar | thanks! | 12:22 |
| weshay|rover | chandankumar did I push back the meeting too late? | 12:23 |
| chandankumar | weshay|rover: how much? | 12:23 |
| weshay|rover | I did it a few days ago | 12:23 |
| quiquell | weshay|rover: o/ what was the issue with reproducer ? | 12:23 |
| chandankumar | oh sorry | 12:23 |
| weshay|rover | it's at 2pm utc | 12:23 |
| chandankumar | weshay|rover: it's ok | 12:23 |
| weshay|rover | quiquell containers were not downloading because the image we used was missing a directory | 12:24 |
| weshay|rover | only some of the containers | 12:24 |
| weshay|rover | quiquell in the standalone deployment | 12:24 |
| weshay|rover | not the zuul setup | 12:24 |
| quiquell | weshay|rover: do you have logs ? | 12:24 |
| weshay|rover | sshnaidm|off was working w/ Emilien on it.. they had hit the same issue earlier | 12:24 |
| weshay|rover | quiquell I might | 12:24 |
| * weshay|rover looks | 12:24 | |
| arxcruz | shit, sound was off, didn't saw the notifications | 12:25 |
| arxcruz | chandankumar: sure | 12:25 |
| arxcruz | marios_|ruck: quiquell sshnaidm|off weshay|rover https://review.openstack.org/#/c/648110/ | 12:26 |
| arxcruz | weshay|rover: chandankumar kopecmartin you guys want to do the meeting earlier ? | 12:26 |
| hubbot1 | FAILING CHECK JOBS on master: tripleo-ci-centos-7-standalone-upgrade, tripleo-ci-centos-7-scenario012-standalone, tripleo-ci-centos-7-scenario008-multinode-oooq-container, tripleo-ci-centos-7-scenario010-multinode-oooq-container @ https://review.openstack.org/604298, stable/pike: tripleo-ci-centos-7-ovb-3ctlr_1comp_1supp-featureset039 @ https://review.openstack.org/602248, master: tripleo-ci-centos-7 (2 more messages) | 12:28 |
| arxcruz | marios_|ruck: do you remember the args i need to pass to change the ntp server ? | 12:29 |
| kopecmartin | chandankumar, arxcruz, up to you, i'm flexible, we can start 30 or 20 minutes earlier if you want | 12:29 |
| marios_|ruck | arxcruz: you mean on beaker box ? | 12:29 |
| arxcruz | marios_|ruck: yes | 12:29 |
| marios_|ruck | arxcruz: or in what context | 12:29 |
| marios_|ruck | arxcruz: sec | 12:29 |
| marios_|ruck | arxcruz: notes here https://tree.taiga.io/project/tripleo-ci-board/task/708?kanban-status=1447277 | 12:30 |
| marios_|ruck | arxcruz: lemme know if you don't find what you want | 12:31 |
| quiquell | weshay|rover: ahh the stat /dev/log: no such file or directory | 12:31 |
| *** jpena is now known as jpena|lunch | 12:32 | |
| arxcruz | marios_|ruck: thanks, added a comment there, there are two variables you must edit | 12:34 |
| arxcruz | -e ntp_server=clock.redhat.com -e undercloud_ntp_servers=clock.redhat.com | 12:34 |
| marios_|ruck | arxcruz: tx | 12:36 |
| *** holser_ has joined #oooq | 12:45 | |
| *** jtomasek has joined #oooq | 12:47 | |
| quiquell | rfolco: kolla failing at fedora28 stein build :-/ | 12:48 |
| arxcruz | weshay|rover: chandankumar kopecmartin can we do the meeting in 10 ? | 12:50 |
| kopecmartin | sure | 12:50 |
| weshay|rover | quiquell can I push out your 1-1? | 12:52 |
| quiquell | weshay|rover: we can review the performance stuff whenever you have time | 12:54 |
| *** quiquell is now known as quiquell|lunch | 12:56 | |
| *** jaosorior has joined #oooq | 12:56 | |
| quiquell|lunch | weshay|rover: have complet it though | 12:56 |
| quiquell|lunch | weshay|rover: or this is what I think | 12:56 |
| quiquell|lunch | weshay|rover: btw I have being trying upgrade jobs for queens at reproducer | 12:57 |
| quiquell|lunch | weshay|rover: there is work to do there | 12:57 |
| quiquell|lunch | Tengu: trying to with me | 12:58 |
| weshay|rover | nice | 12:58 |
| Tengu | ;_; | 12:59 |
| Tengu | has anybody seen my brain? it's apparently flying away | 12:59 |
| rfolco | quiquell|lunch, yes, I'm looking at the logs | 13:00 |
| weshay|rover | chandankumar let me know when you guys start | 13:02 |
| weshay|rover | arxcruz ^ | 13:03 |
| chandankumar | weshay|rover: yes | 13:04 |
| chandankumar | kopecmartin: arxcruz weshay|rover https://redhat.bluejeans.com/1571313919/ | 13:04 |
| quiquell|lunch | rfolco: anything else to review from rdo? | 13:05 |
| chandankumar | weshay|rover: we are waiting for you! | 13:06 |
| rfolco | quiquell|lunch, I don't think so, will just try to fix f28 now | 13:07 |
| quiquell|lunch | rfolco: ack, have some comments o. Th | 13:07 |
| quiquell|lunch | On the depends on of the testproject | 13:07 |
| chandankumar | arxcruz: kopecmartin for handling tempest plugins installation in os_tempest https://review.openstack.org/#/q/topic:ostempest_plugins+(status:open+OR+status:merged) | 13:07 |
| chandankumar | using featureset override | 13:07 |
| chandankumar | weshay|rover: https://tree.taiga.io/project/tripleo-ci-board/task/898 | 13:10 |
| chandankumar | weshay|rover: https://tree.taiga.io/project/tripleo-ci-board/us/867 | 13:16 |
| chandankumar | weshay|rover: arxcruz https://review.openstack.org/#/q/topic:os_tempest+(status:open+OR+status:merged) | 13:17 |
| marios_|ruck | quiquell|lunch: is there some known thing. i had that last time i tried like last friday http://pastebin.test.redhat.com/742727 | 13:18 |
| quiquell|lunch | marios_|ruck: don't have vpn here can you paste it upstream pastebin? | 13:19 |
| rfolco | quiquell|lunch, whats wrong with the depends-on? | 13:21 |
| marios_|ruck | quiquell|lunch: http://paste.openstack.org/show/748581/ | 13:22 |
| quiquell|lunch | marios_|ruck: ssh keys are good? | 13:22 |
| quiquell|lunch | rfolco: missing elif for non centos | 13:23 |
| marios_|ruck | quiquell|lunch: yeah i think so i generated them new ones but anyway i will try again with fresh vm was just asking in case | 13:25 |
| rfolco | quiquell|lunch, please comment on the patch | 13:25 |
| quiquell|lunch | Let's tmate it after lunch | 13:25 |
| quiquell|lunch | rfolco: is a comment there | 13:25 |
| *** rlandy has joined #oooq | 13:26 | |
| rfolco | quiquell|lunch, tmate me or marios_|ruck | 13:26 |
| marios_|ruck | rfolco: ? | 13:28 |
| rfolco | marios_|ruck, crossed chats | 13:28 |
| rfolco | marios_|ruck, ignore for now | 13:28 |
| marios_|ruck | rfolco: ack | 13:28 |
| quiquell|lunch | marios_|ruck | 13:28 |
| rfolco | there is, he wants to chat with you marios_|ruck | 13:28 |
| rfolco | nobody tmates me | 13:28 |
| rfolco | :( | 13:29 |
| marios_|ruck | wut | 13:29 |
| *** vinaykns has joined #oooq | 13:29 | |
| chandankumar | arxcruz: weshay|rover: https://review.openstack.org/#/c/648373/ | 13:30 |
| *** jaosorior has quit IRC | 13:30 | |
| arxcruz | weshay|rover: chandankumar https://review.openstack.org/#/c/648110/ and https://review.openstack.org/#/c/648111/ | 13:30 |
| chandankumar | arxcruz: weshay|rover https://review.openstack.org/#/q/topic:ostempest_plugins+(status:open+OR+status:merged) | 13:31 |
| marios_|ruck | weshay|rover: rfolco fyi picked up scen7 cos i posted patches loong time ago rebased updated stuff see https://tree.taiga.io/project/tripleo-ci-board/task/931 && /932 | 13:32 |
| marios_|ruck | don't know how far i'll get by end of sprint but we'll see | 13:32 |
| marios_|ruck | job was red last time i tried | 13:33 |
| marios_|ruck | running now at https://review.openstack.org/#/c/631497/3 | 13:33 |
| rfolco | marios_|ruck, you rock | 13:33 |
| marios_|ruck | rfolco: no no | 13:33 |
| marios_|ruck | rfolco: this is just rebase now | 13:33 |
| marios_|ruck | rfolco: anyway we'll see | 13:33 |
| rfolco | marios_|ruck, you rock and roll | 13:33 |
| rfolco | :) | 13:33 |
| *** vinaykns has quit IRC | 13:33 | |
| marios_|ruck | rfolco: we all do man we're awesome happy friday now get back to work ! | 13:33 |
| *** jtomasek has quit IRC | 13:34 | |
| *** jtomasek has joined #oooq | 13:34 | |
| *** jpena|lunch is now known as jpena | 13:35 | |
| *** jtomasek has quit IRC | 13:36 | |
| chandankumar | weshay|rover: arxcruz http://codesearch.openstack.org/?q=use_os_tempest&i=nope&files=.yaml&repos= | 13:36 |
| *** Vorrtex has joined #oooq | 13:36 | |
| rlandy | rfolco: hi - anything still requiring review/merge? | 13:42 |
| *** jaosorior has joined #oooq | 13:43 | |
| chandankumar | arxcruz: weshay|rover https://github.com/openstack/openstack-ansible-os_ironic/blob/master/tests/test-ironic-cli.yml | 13:44 |
| rfolco | rlandy, we're all good except f28 I think | 13:45 |
| rlandy | rfolco: k - doing a list review of your bm doc | 13:46 |
| rfolco | rlandy, oh yes, bm review.... | 13:46 |
| rfolco | btw, I tested with ipmi driver and it worked apparently | 13:46 |
| rfolco | rlandy, ^ | 13:46 |
| rlandy | k - good to know | 13:46 |
| rfolco | rlandy, dtantsur suggested on the doc to switch to "ipmi" from pxe_impitool | 13:47 |
| rlandy | rfolco;l - so you can out up a patch to change that | 13:47 |
| rlandy | rfolco: just fyi - we should do this after the move | 13:47 |
| rfolco | rlandy, yep, will double check if all good, and then change all instackenv.json | 13:47 |
| rfolco | yep yep | 13:48 |
| rfolco | rlandy, thanks for reviewing doc | 13:48 |
| rlandy | rfolco: I need to start shutting down envs this afternoon | 13:48 |
| rlandy | sure | 13:48 |
| rlandy | rfolco: overall doc is good - +2'ed - just a few comments in line https://review.openstack.org/#/c/648206 (typos) | 13:50 |
| rfolco | rlandy, ok, give me 5 min and do a +2 again... will fix this now | 13:51 |
| rfolco | rlandy, done | 13:54 |
| rfolco | 3 min | 13:54 |
| rfolco | :) | 13:54 |
| rlandy | k | 13:54 |
| rlandy | done | 13:55 |
| arxcruz | weshay|rover: https://github.com/openstack/tripleo-quickstart-extras/blob/master/roles/validate-tempest/templates/cleanup-network.sh.j2#L51-L80 | 13:56 |
| rfolco | rlandy, thx | 13:57 |
| rlandy | quiquell|lunch: I think we should disable the periodic-hourly today ahead of the hardware move | 13:57 |
| quiquell|lunch | rlandy: does not make sense to have it | 13:57 |
| quiquell|lunch | rlandy: also we are missing the image build | 13:57 |
| rlandy | quiquell|lunch: I am fixing that now | 13:59 |
| rlandy | quiquell|lunch: building images in the job like ovb | 13:59 |
| marios_|ruck | arxcruz: chandankumar you folks around? | 14:00 |
| *** jaosorior has quit IRC | 14:01 | |
| *** amoralej is now known as amoralej|lunch | 14:01 | |
| *** quiquell|lunch is now known as quiquell | 14:02 | |
| quiquell | rlandy: ack | 14:02 |
| *** jaosorior has joined #oooq | 14:02 | |
| quiquell | marios_|ruck: still issues with repro | 14:02 |
| marios_|ruck | quiquell: thanks trying to get this tempest thing cleared up in tripleo now i'll maybe try later didn't even start yet | 14:05 |
| quiquell | ok | 14:06 |
| zbr | rlandy: a f28 nit if you can https://review.openstack.org/#/c/648158/ - thanks. | 14:06 |
| quiquell | zbr: +w | 14:07 |
| rlandy | quiquell beat me to it | 14:07 |
| quiquell | I like to merge stuff on friday | 14:08 |
| weshay|rover | marios_|ruck we're in a call for tempest atm | 14:11 |
| weshay|rover | what's up | 14:11 |
| rlandy | quiquell: one more for your friday merge happy attitude https://code.engineering.redhat.com/gerrit/166526 Temp disable hourly pipeline while hardware moves. | 14:12 |
| quiquell | rlandy: this reduce moving parts, merged | 14:13 |
| chandankumar | weshay|rover: https://bugzilla.redhat.com/show_bug.cgi?id=1689753 | 14:14 |
| openstack | bugzilla.redhat.com bug 1689753 in openstack-tempest "Unable to access dashboard as soon as tempest tests are executed on overcloud" [Urgent,New] - Assigned to chkumar | 14:14 |
| quiquell | Tengu: this rings any bell ? http://logs.openstack.org/19/647419/9/check/tripleo-ci-centos-7-standalone-upgrade/c12d155/job-output.txt.gz#_2019-03-29_12_33_34_735195 | 14:17 |
| quiquell | Tengu: is related to validate failed containers | 14:17 |
| Tengu | quiquell: apparently it did match failed container ? | 14:17 |
| Tengu | uh ? | 14:18 |
| Tengu | wait.... is this run as a non-root in some cases? | 14:18 |
| quiquell | Ahh that's right | 14:18 |
| Tengu | ... | 14:18 |
| Tengu | inconsistency.... | 14:18 |
| Tengu | -.- I love that. | 14:18 |
| quiquell | this is running with stein | 14:18 |
| quiquell | is a standalone upgrade from stain->master | 14:19 |
| Tengu | and there, non-root is running the checks? | 14:19 |
| quiquell | but containers haven't being tested yet so they can be wrong | 14:19 |
| Tengu | quiquell: the error "cannot re-exec process" is usually due to "non-root trying to run podman on centos7 kernel" | 14:20 |
| quiquell | Let me check master standalone run | 14:21 |
| *** vinaykns has joined #oooq | 14:21 | |
| quiquell | Thisinstall standalone with the new stein release file | 14:21 |
| marios_|ruck | weshay|rover: ack yeah i gathered thanks chandankumar kopecmartin arxcruz when you have some time please check comment #8 n https://bugs.launchpad.net/tripleo/+bug/1822080 | 14:21 |
| openstack | Launchpad bug 1822080 in tripleo "periodic centos-7-ovb-1ctlr_1comp-featureset020-queens failing tempest "Endpoint not found"" [Critical,Triaged] - Assigned to Martin Kopec (mkopec) | 14:21 |
| quiquell | master is skipping validate services | 14:23 |
| Tengu | quiquell: do you need me on that? else I think I'll call it a day/week and try to catch my brain :) | 14:26 |
| quiquell | Tengu: catch that brain you need it for the weekend | 14:26 |
| Tengu | hehe :) fine | 14:26 |
| Tengu | thanks again for your time! | 14:26 |
| rfolco | quiquell, do you have any ideas on what could be the issue here ? http://logs.rdoproject.org/19/19819/5/check/periodic-tripleo-fedora-28-stein-containers-build-push/ce7bef1/logs/build.log.txt.gz | 14:27 |
| hubbot1 | FAILING CHECK JOBS on master: tripleo-ci-centos-7-standalone-upgrade, tripleo-ci-centos-7-scenario012-standalone, tripleo-ci-centos-7-scenario008-multinode-oooq-container, tripleo-ci-centos-7-scenario010-multinode-oooq-container @ https://review.openstack.org/604298, stable/pike: tripleo-ci-centos-7-ovb-3ctlr_1comp_1supp-featureset039 @ https://review.openstack.org/602248, master: tripleo-ci-centos-7 (2 more messages) | 14:28 |
| quiquell | rfolco: I think stein has break dependencies at fedora28 | 14:28 |
| quiquell | rfolco: Look for "Error:" | 14:28 |
| quiquell | 2019-03-29 11:16:19 | INFO:kolla.common.utils.tripleoclient: Problem: package python3-tripleoclient-11.3.1-0.20190328065619.0132e7d.fc28.noarch requires openstack-tripleo-common >= 10.0.0, but none of the providers can be installed | 14:29 |
| quiquell | 2019-03-29 11:16:19 | INFO:kolla.common.utils.tripleoclient: - conflicting requests | 14:29 |
| quiquell | 2019-03-29 11:16:19 | INFO:kolla.common.utils.tripleoclient: - nothing provides golang-github-vbatts-tar-split >= 0.11.1 needed by openstack-tripleo-common-10.6.1-0.20190328204902.02f5c67.fc28.noarch | 14:29 |
| quiquell | 2019-03-29 11:16:19 | INFO:kolla.common.utils.tripleoclient:[0m | 14:29 |
| weshay|rover | quiquell 1-1 | 14:29 |
| quiquell | weshay|rover: ack | 14:29 |
| kopecmartin | arxcruz, chandankumar when you have a moment, please: https://review.openstack.org/#/c/647415/ and https://review.openstack.org/#/c/647696/ | 14:31 |
| weshay|rover | quiquell sudo ln -s /run/systemd/journal/dev-log /dev/log | 14:54 |
| weshay|rover | sudo apt-get install -y rsyslog | 14:55 |
| *** aakarsh has quit IRC | 14:55 | |
| *** amoralej|lunch is now known as amoralej | 15:00 | |
| *** ratailor has joined #oooq | 15:00 | |
| quiquell | weshay|rover: I am going to push upstream imges to my tenant hacking them before so I can use them for repro | 15:01 |
| quiquell | weshay|rover: and see if then they work | 15:01 |
| *** aakarsh has joined #oooq | 15:15 | |
| *** aakarsh|2 has joined #oooq | 15:16 | |
| marios_|ruck | ack kopecmartin thanks for input on the bug | 15:19 |
| marios_|ruck | kopecmartin: mostly happy that there is at least some focus on that | 15:19 |
| kopecmartin | marios_|ruck, no problem, now the hard part, we need to find the review :D | 15:19 |
| marios_|ruck | kopecmartin: are you trying to find it? | 15:19 |
| *** aakarsh has quit IRC | 15:19 | |
| kopecmartin | yes | 15:20 |
| marios_|ruck | kopecmartin: ack thanks i'll leave you be | 15:20 |
| marios_|ruck | kopecmartin: i mean, even if it idoesn't happen today, still have the option to revert, unblock. then you can find the fix without pressure | 15:20 |
| marios_|ruck | anyway | 15:20 |
| marios_|ruck | wel'll see what others think | 15:20 |
| *** jaosorior has quit IRC | 15:21 | |
| *** skramaja_ has quit IRC | 15:21 | |
| chandankumar | quiquell: marios_|ruck weshay|rover https://review.openstack.org/#/c/648373/ and https://review.openstack.org/#/c/648617/ when free! | 15:21 |
| marios_|ruck | chandankumar: no | 15:23 |
| marios_|ruck | chandankumar: N E V E R | 15:23 |
| chandankumar | marios_|ruck: I think I got the votes! | 15:23 |
| *** gouthamr has quit IRC | 15:24 | |
| *** dmellado has quit IRC | 15:24 | |
| chandankumar | marios_|ruck: but why? | 15:25 |
| chandankumar | sometime people spell why to Bhai in Hindi | 15:25 |
| *** dmellado has joined #oooq | 15:26 | |
| marios_|ruck | chandankumar: because https://www.youtube.com/watch?v=dQw4w9WgXcQ | 15:26 |
| chandankumar | marios_|ruck: :-) | 15:26 |
| *** gouthamr has joined #oooq | 15:27 | |
| chandankumar | see ya guys, Happy weekend :-) | 15:30 |
| *** chandankumar is now known as raukadah | 15:30 | |
| marios_|ruck | have a good one raukadah ttyl | 15:30 |
| *** kopecmartin is now known as kopecmartin|off | 15:32 | |
| rlandy | rfolco: ping re: https://review.rdoproject.org/r/#/c/19580/ | 15:33 |
| rlandy | rfolco: https://tree.taiga.io/project/tripleo-ci-board/task/885?kanban-status=1447276 stated I should add multinode jobs | 15:33 |
| rlandy | which I did in the review | 15:33 |
| rlandy | which nobody touched | 15:34 |
| rlandy | and then was merged in another review | 15:34 |
| rlandy | rfolco: pls review https://review.rdoproject.org/r/#/c/19580/ | 15:34 |
| rlandy | and let me know if I can merge | 15:34 |
| *** sshnaidm|off has quit IRC | 15:34 | |
| rlandy | also - I need to know if this card includes adding any of these stein jobs to templates | 15:34 |
| *** aakarsh|2 is now known as aakarsh | 15:36 | |
| rfolco | rlandy, looking | 15:40 |
| *** ykarel has quit IRC | 15:40 | |
| *** sshnaidm|off has joined #oooq | 15:48 | |
| rfolco | rlandy, my cards included periodic + branchful... for some reason there is dup of tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset001-stein-vexxhost | 15:54 |
| rfolco | rlandy, pls see my comment | 15:54 |
| quiquell | rfolco, weshay|rover: WIP about using promoter with fedora https://review.rdoproject.org/r/#/c/19846/ | 15:56 |
| *** ratailor has quit IRC | 15:56 | |
| quiquell | rlandy: ^ and also https://review.rdoproject.org/r/#/c/19846/ to paremterize container-push | 15:57 |
| quiquell | droping now | 15:57 |
| rfolco | quiquell, if you get any ideas on how to fix dependencies/conflicts for f28 containers build let me know | 15:57 |
| quiquell | Have a good weekend | 15:57 |
| rfolco | quiquell, missing golang-github... for example | 15:57 |
| quiquell | rfolco: have some time | 15:57 |
| quiquell | rfolco: let's do some forensics on it | 15:57 |
| quiquell | rfolco: can you paste the pointer ? | 15:58 |
| quiquell | rfolco: could be that something is missing at RDO distgit | 15:58 |
| quiquell | rfolco: stein+fedora28 has to be very alpha | 15:58 |
| rfolco | quiquell, will talk to weshay|rover about it... have a good one | 15:58 |
| quiquell | ack | 15:58 |
| *** trown is now known as trown|lunch | 15:58 | |
| quiquell | rfolco: jpena and amoralej can be of help there | 15:59 |
| quiquell | rfolco: ok read you next week | 15:59 |
| *** quiquell is now known as quiquell|off | 15:59 | |
| rlandy | rfolco: https://review.rdoproject.org/r/#/c/19580/ updated and comments answered | 16:00 |
| rfolco | rlandy, are you sure we need to create a new branchful job for fs035 ? We don't have the same for rocky... for all changes in master, we'll run this -stein job. Is that what was defined in the design ? https://review.rdoproject.org/r/#/c/19580/5/zuul.d/ovb-jobs.yaml@57 | 16:08 |
| rlandy | rfolco: idk - you guys created the card | 16:08 |
| rfolco | weshay|rover, can you please look at https://review.rdoproject.org/r/#/c/19580/5/zuul.d/ovb-jobs.yaml@57 | 16:09 |
| rfolco | weshay|rover, rlandy IMO fs035 on stein a no-op change, as the regex does not block stein | 16:09 |
| weshay|rover | k.. in a few | 16:10 |
| marios_|ruck | weshay|rover: quiquell|off rlandy i keep hitting that - its rdo vm http://paste.openstack.org/show/748594/ - have hit it in the past. this time i did a yum update and reboot before starting, but *still* hit it. yum update gave me python2-passlib have no idea if that is related. anyway rerun clears it at least now its on starting zuul and friends | 16:14 |
| marios_|ruck | weshay|rover: quiquell|off rlandy definitely hit it before i documented it here first time issue 1 at https://tree.taiga.io/project/tripleo-ci-board/task/765 | 16:14 |
| weshay|rover | marios_|ruck ya.. I tried fixing that w/ an ansible retry | 16:15 |
| weshay|rover | what I put in did not work.. | 16:15 |
| rfolco | weshay|rover, 1x1 knock | 16:15 |
| marios_|ruck | doh /me has 47 tries to update the ssh keys before it fails | 16:16 |
| marios_|ruck | now 46.. | 16:16 |
| weshay|rover | rfolco is it possible to delay by 10min.. I'm finishing off the perf/devel doc | 16:16 |
| rfolco | weshay|rover, np | 16:16 |
| weshay|rover | marios_|ruck that error is just the connection to rdo-cloud | 16:16 |
| marios_|ruck | weshay|rover: ah | 16:16 |
| weshay|rover | so we need to make that more resiliant | 16:16 |
| marios_|ruck | well need the keys anyway | 16:16 |
| weshay|rover | it's just trying to pull the image info | 16:17 |
| marios_|ruck | weshay|rover: the problem is the jinja version ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDWaYLdTAosPQt4m8k0LXBTNs6cZN3UKHSuCulsq1E0MhJ7IArIK1YBKPM+VZJLAe8iCkKS99XNXcxBiLLDeXvjKGB8Df+MhsKZJRtqn2X+pUs+hMZnUiENVqpcHXgZujtq4B5mI7f0p2+JC/7SUyNUYX4I8DzdNDYFbP8oQ3+Eu1LWEedhVT7gLcW7pV6CwYtHw9LJpZIIYpjInAb7oh31X/BBDKfkwSsnTUkD4EwMAprtZIg2YiAoJPaAr3yYUbabs9+hJ4QhYiVn3OKhuOOdIKURabm+Z08a77+Ygu/k6gyCZdp/VRxujG5ChOrS+t4e+21Y4LrCJekHWaAlrdSz reprozuul@reproz | 16:17 |
| marios_|ruck | weshay|rover: hah thats the pub key sorry | 16:17 |
| marios_|ruck | weshay|rover: https://access.redhat.com/solutions/3401041 here | 16:18 |
| marios_|ruck | weshay|rover: if there is a way to do it without equalto | 16:18 |
| marios_|ruck | weshay|rover: anyway i keep hitting it so hence my mention | 16:18 |
| marios_|ruck | rerun clears it | 16:18 |
| marios_|ruck | weirdly | 16:18 |
| marios_|ruck | so i don't see how jinja version matters | 16:19 |
| marios_|ruck | :( | 16:19 |
| marios_|ruck | who am i | 16:19 |
| marios_|ruck | weshay|rover: any update from panda? | 16:19 |
| marios_|ruck | i only have work email i thought i had him on hangouts but dont | 16:19 |
| marios_|ruck | weshay|rover: i tried that but don't know if he is reading | 16:19 |
| *** tosky has quit IRC | 16:23 | |
| weshay|rover | marios_|ruck no not yet.. he's still sick and getting tests done | 16:27 |
| weshay|rover | rfolco ready for 1-1 | 16:27 |
| hubbot1 | FAILING CHECK JOBS on master: tripleo-ci-centos-7-standalone-upgrade, tripleo-ci-centos-7-scenario012-standalone, tripleo-ci-centos-7-scenario008-multinode-oooq-container, tripleo-ci-centos-7-scenario010-multinode-oooq-container @ https://review.openstack.org/604298, stable/pike: tripleo-ci-centos-7-ovb-3ctlr_1comp_1supp-featureset039 @ https://review.openstack.org/602248, master: tripleo-ci-centos-7 (2 more messages) | 16:28 |
| rfolco | weshay|rover, joining | 16:28 |
| *** rlandy is now known as rlandy|brb | 16:30 | |
| marios_|ruck | weshay|rover: initiating shutdown sequence, latest things are in https://etherpad.openstack.org/p/ruckroversprint7 and especially please consider comment #8 @ https://bugs.launchpad.net/tripleo/+bug/1822080 to unblock queens we are yellow there still for that it hits a lot of the queens jobs probably all at least 3 as per the bug | 16:30 |
| openstack | Launchpad bug 1822080 in tripleo "periodic centos-7-ovb-1ctlr_1comp-featureset020-queens failing tempest "Endpoint not found"" [Critical,Triaged] - Assigned to Martin Kopec (mkopec) | 16:30 |
| weshay|rover | marios_|ruck thank you! will look at the bug in a sec | 16:31 |
| marios_|ruck | weshay|rover: otherwise master/rocky/pike good afaics and promoted today | 16:31 |
| marios_|ruck | afaics | 16:31 |
| *** rlandy|brb is now known as rlandy | 16:49 | |
| *** ykarel has joined #oooq | 16:59 | |
| marios_|ruck | ttyl folks /me out | 17:01 |
| *** ykarel has quit IRC | 17:03 | |
| *** dtantsur is now known as dtantsur|afk | 17:06 | |
| *** bogdando has quit IRC | 17:16 | |
| rlandy | weshay|rover: hardware shutdown starting - beginning with envs A, B, 1, E | 17:19 |
| rlandy | I will do the personal boxes last | 17:20 |
| rlandy | still testing on env D today | 17:20 |
| rlandy | I notified dtrainor that he should shut down his boxes - not touching those | 17:20 |
| *** jpena is now known as jpena|off | 17:37 | |
| weshay|rover | k :) | 17:48 |
| *** yolanda has quit IRC | 17:53 | |
| *** sshnaidm|off has quit IRC | 18:08 | |
| *** trown|lunch is now known as trown | 18:08 | |
| zbr | weshay|rover: how are with your time? | 18:08 |
| *** amoralej is now known as amoralej|off | 18:11 | |
| rfolco | amoralej, Apparently stein consistent is not good for f28, I've got several package conflicts... can you please take a look ? I am trying to build stein containers for f28 and this is showstopper - http://logs.rdoproject.org/19/19819/5/check/periodic-tripleo-fedora-28-stein-containers-build-push/ce7bef1/logs/build.log.txt.gz | 18:11 |
| raukadah | rlandy: zbr if around need some help on this review https://review.openstack.org/#/c/648678/2/playbooks/tempest.yml | 18:18 |
| raukadah | I need set some var key dynamically | 18:18 |
| raukadah | testing stuff here https://review.openstack.org/#/c/648680/3/zuul.d/standalone-jobs.yaml | 18:19 |
| rlandy | looking | 18:20 |
| *** sshnaidm|off has joined #oooq | 18:21 | |
| raukadah | zbr: rlandy thanks@ | 18:24 |
| *** vinaykns has quit IRC | 18:28 | |
| hubbot1 | FAILING CHECK JOBS on master: tripleo-ci-centos-7-standalone-upgrade, tripleo-ci-centos-7-scenario012-standalone, tripleo-ci-centos-7-scenario008-multinode-oooq-container, tripleo-ci-centos-7-scenario010-multinode-oooq-container @ https://review.openstack.org/604298, stable/pike: tripleo-ci-centos-7-ovb-3ctlr_1comp_1supp-featureset039 @ https://review.openstack.org/602248, master: tripleo-ci-centos-7 (2 more messages) | 18:28 |
| rlandy | raukadah: http://logs.openstack.org/78/648678/2/check/tripleo-ci-centos-7-standalone-os-tempest/e1a1218/ fails | 18:29 |
| rlandy | is there a depends-on change to make this pass? | 18:29 |
| raukadah | rlandy: I am doing depends on toci patch with tqe patch | 18:29 |
| raukadah | rlandy: toci patch passed but functionality does not worked | 18:30 |
| raukadah | but I will improve that part on zbr and yours feedback | 18:30 |
| rlandy | raukadah: as things stand, if we merge https://review.openstack.org/#/c/648678/, tripleo-ci-centos-7-standalone-os-tempest/e1a1218/ fails | 18:31 |
| zbr | raukadah: do i need to get used to the new irc nick? | 18:31 |
| raukadah | zbr: you need to swap always looks for freenode registered nick, all are grouped with a single nick | 18:32 |
| raukadah | chandan_kumar | 18:32 |
| raukadah | there are multiple | 18:32 |
| raukadah | chandankumar, chkumar, chkumar246, raukadah | 18:32 |
| zbr | ok | 18:33 |
| * raukadah adding a habit to swap the nick to remind myself to enjoy other part of life | 18:33 | |
| weshay|rover | raukadah what is life? | 18:34 |
| rlandy | raukadah: left comment | 18:34 |
| zbr | whatever makes you happy :) only that in my native language "rau" translates to "bad" :) | 18:34 |
| rlandy | if there is a way to get by the failure by a default/empty/dummy definition, I'll +2 it | 18:34 |
| raukadah | weshay|rover: interesting question, life is something where we cherrish to experience the happiness & sarrow by experimenting most of the stuff | 18:35 |
| weshay|rover | raukadah are you telling me to do drugs? | 18:36 |
| raukadah | weshay|rover: no, never do that | 18:36 |
| weshay|rover | lolz | 18:36 |
| raukadah | stuff ~= undefined | 18:36 |
| dtrainor | rlandy ack | 18:44 |
| *** raukadah has quit IRC | 18:45 | |
| arxcruz | weshay|rover: hey boss, where's that doc showing how to run quickstart.sh ? | 18:49 |
| *** chem has quit IRC | 18:49 | |
| *** chem has joined #oooq | 18:50 | |
| weshay|rover | arxcruz https://docs.openstack.org/tripleo-quickstart/latest/ | 18:52 |
| weshay|rover | arxcruz in $subject | 18:52 |
| *** matbu has quit IRC | 19:10 | |
| *** tosky has joined #oooq | 19:48 | |
| *** Vorrtex has quit IRC | 19:55 | |
| *** yolanda has joined #oooq | 19:58 | |
| *** matbu has joined #oooq | 20:07 | |
| *** jtomasek has joined #oooq | 20:17 | |
| *** holser_ has quit IRC | 20:25 | |
| *** jtomasek has quit IRC | 20:26 | |
| hubbot1 | FAILING CHECK JOBS on master: tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset001, tripleo-ci-centos-7-standalone-upgrade, tripleo-ci-centos-7-scenario012-standalone, tripleo-ci-centos-7-scenario008-multinode-oooq-container, tripleo-ci-centos-7-scenario010-multinode-oooq-container @ https://review.openstack.org/604298, stable/pike: tripleo-ci-centos-7-ovb-3ctlr_1comp_1supp-featureset039, tripleo-ci- (2 more messages) | 20:28 |
| *** chem has quit IRC | 20:51 | |
| *** chem has joined #oooq | 20:54 | |
| *** trown is now known as trown|outtypewww | 21:00 | |
| *** marios_|ruck has quit IRC | 21:16 | |
| *** marios has quit IRC | 21:16 | |
| rlandy | weshay|rover: all hardware is shut down - expect for env D undercloud (test still running) and the two boxes marked unreachable | 21:27 |
| *** holser_ has joined #oooq | 21:49 | |
| *** chem has quit IRC | 21:51 | |
| *** chem has joined #oooq | 21:52 | |
| *** rlandy has quit IRC | 22:02 | |
| hubbot1 | FAILING CHECK JOBS on master: tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset001, tripleo-ci-centos-7-standalone-upgrade, tripleo-ci-centos-7-scenario012-standalone, tripleo-ci-centos-7-scenario008-multinode-oooq-container, tripleo-ci-centos-7-scenario010-multinode-oooq-container @ https://review.openstack.org/604298, stable/pike: tripleo-ci-centos-7-ovb-3ctlr_1comp_1supp-featureset039, tripleo-ci- (2 more messages) | 22:28 |
| *** aakarsh has quit IRC | 22:49 | |
| *** chem has quit IRC | 22:52 | |
| *** chem has joined #oooq | 22:53 | |
Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!