| *** saneax has quit IRC | 00:07 | |
| *** rfolco has joined #oooq | 00:19 | |
| *** rfolco has quit IRC | 01:33 | |
| *** apetrich has quit IRC | 01:58 | |
| *** sshnaidm has quit IRC | 02:05 | |
| *** skramaja has joined #oooq | 02:53 | |
| *** ykarel|afk has joined #oooq | 03:30 | |
| *** udesale has joined #oooq | 03:53 | |
| *** saneax has joined #oooq | 04:07 | |
| *** ykarel|afk is now known as ykarel | 04:23 | |
| *** saneax has quit IRC | 04:25 | |
| *** ykarel has quit IRC | 04:32 | |
| *** ykarel has joined #oooq | 04:48 | |
| *** ratailor has joined #oooq | 04:58 | |
| *** marios has joined #oooq | 05:42 | |
| *** jtomasek has joined #oooq | 05:51 | |
| *** skramaja has quit IRC | 05:57 | |
| *** skramaja has joined #oooq | 05:57 | |
| *** jfrancoa has joined #oooq | 06:05 | |
| *** jfrancoa has quit IRC | 06:11 | |
| *** saneax has joined #oooq | 06:17 | |
| *** jfrancoa has joined #oooq | 06:23 | |
| *** altlogbot_1 has quit IRC | 06:46 | |
| *** altlogbot_1 has joined #oooq | 06:49 | |
| *** altlogbot_1 has quit IRC | 06:50 | |
| *** altlogbot_1 has joined #oooq | 06:54 | |
| *** tesseract has joined #oooq | 07:11 | |
| *** dsneddon has joined #oooq | 07:25 | |
| *** amoralej|off is now known as amoralej | 07:27 | |
| *** amoralej is now known as amoralej|off | 07:27 | |
| *** bogdando has joined #oooq | 07:28 | |
| *** tosky has joined #oooq | 07:28 | |
| *** saneax has quit IRC | 07:31 | |
| *** jtomasek has quit IRC | 07:31 | |
| *** saneax has joined #oooq | 07:32 | |
| *** jtomasek has joined #oooq | 07:32 | |
| *** ykarel is now known as ykarel|lunch | 07:41 | |
| *** ccamacho has joined #oooq | 07:41 | |
| *** ccamacho has quit IRC | 07:41 | |
| *** sshnaidm has joined #oooq | 07:42 | |
| *** ccamacho has joined #oooq | 07:43 | |
| *** jpena|off is now known as jpena | 07:48 | |
| *** jpena is now known as jpena|mtg | 07:48 | |
| chandankumar | always use yaml.safe_dump instead of yaml.dump to avoid unicode issues | 07:48 |
|---|---|---|
| *** apetrich has joined #oooq | 07:54 | |
| *** udesale has quit IRC | 08:01 | |
| *** udesale has joined #oooq | 08:03 | |
| *** udesale has quit IRC | 08:03 | |
| *** udesale has joined #oooq | 08:03 | |
| arxcruz | sshnaidm, https://review.rdoproject.org/r/#/c/21212/ | 08:09 |
| arxcruz | sshnaidm, if you can approve I appreciate :) | 08:09 |
| arxcruz | sshnaidm, and here's my problem with the ovb job on reproducer https://review.rdoproject.org/r/#/c/21250/ | 08:09 |
| sshnaidm | arxcruz, looking | 08:09 |
| arxcruz | danke | 08:09 |
| sshnaidm | arxcruz, ovb job won't work that way :) | 08:14 |
| arxcruz | sshnaidm, show me the way master :) | 08:14 |
| sshnaidm | arxcruz, I'm afraid it requires much more superpowers.. | 08:15 |
| sshnaidm | arxcruz, if it's possible atm at all | 08:15 |
| arxcruz | sshnaidm, even as a 1ctrl_1comp ? | 08:15 |
| arxcruz | or just 1ctrl ? | 08:15 |
| sshnaidm | arxcruz, it doesn't matter | 08:16 |
| arxcruz | not sure exactly what is missing if you can explain me... | 08:16 |
| sshnaidm | arxcruz, check out ovb reproducer zuul.yaml | 08:16 |
| sshnaidm | arxcruz, did you run ovb with new reproducer? | 08:16 |
| *** holser_ has joined #oooq | 08:17 | |
| arxcruz | sshnaidm, not yet | 08:18 |
| arxcruz | i'll add it on my todo list | 08:19 |
| arxcruz | sshnaidm, you mean, the zuul-runner ? | 08:19 |
| sshnaidm | arxcruz, no, the current one | 08:19 |
| arxcruz | sshnaidm, I don't recall use it | 08:19 |
| sshnaidm | arxcruz, try to use it for ovb, then let's see how we can do it.. | 08:19 |
| *** ykarel|lunch is now known as ykarel | 08:40 | |
| *** chem has joined #oooq | 08:42 | |
| sshnaidm | arxcruz, I think it's worth to have tripleo-ci-reproducer-centos-7-openstack-standalone-dryrun periodic job | 08:47 |
| sshnaidm | arxcruz, maybe this one you meant when talking about ovb? | 08:47 |
| sshnaidm | arxcruz, or you need to create a strict ovb job? where is the task? | 08:48 |
| arxcruz | sshnaidm, weshay wants full reproducer on ovb | 08:48 |
| sshnaidm | arxcruz, do you have a task? | 08:49 |
| arxcruz | sshnaidm, yes, let me grab it for you | 08:49 |
| arxcruz | sshnaidm, https://tree.taiga.io/project/tripleo-ci-board/task/1137?kanban-status=1447275 | 08:50 |
| *** jaosorior has joined #oooq | 09:03 | |
| arxcruz | sshnaidm, ohhh, it's going to need a clouds_secret variable | 09:05 |
| arxcruz | sshnaidm, yeah, i don't know how we are going to have it ... | 09:06 |
| arxcruz | do the vm that run's it, have these secrets ? | 09:06 |
| *** yolanda has joined #oooq | 09:07 | |
| sshnaidm | arxcruz, if you don't set them - no | 09:09 |
| arxcruz | sshnaidm, but how the other jobs are creating the vm | 09:09 |
| sshnaidm | arxcruz, I think we need to discuss it, firstly you need a regular ovb job, not periodic | 09:09 |
| sshnaidm | arxcruz, I don't follow | 09:09 |
| arxcruz | sshnaidm, the openstack-vexxhost creates a vm and deploy tripleo | 09:10 |
| sshnaidm | arxcruz, it creates libvirt | 09:10 |
| sshnaidm | arxcruz, you need to run reproducer on ovb and on libvirt before doing this | 09:11 |
| arxcruz | oh, yeah, it uses libvirt, forgot about that | 09:11 |
| *** jaosorior has quit IRC | 09:11 | |
| arxcruz | shit :/ | 09:11 |
| *** ratailor_ has joined #oooq | 09:14 | |
| *** ratailor has quit IRC | 09:17 | |
| arxcruz | let me create a vm to run this reproducer | 09:17 |
| arxcruz | silverblue don't have docker anymore | 09:17 |
| chandankumar | ykarel: marios: zbr|ruck sshnaidm success on overcloud https://logs.rdoproject.org/86/664586/17/openstack-check/tripleo-ci-centos-7-multinode-1ctlr-featureset010/54c6eab/logs/undercloud/home/zuul/.config/openstack/clouds.yaml.txt.gz | 09:46 |
| ykarel | nice | 09:46 |
| chandankumar | https://logs.rdoproject.org/04/665604/10/openstack-check/tripleo-ci-centos-7-multinode-1ctlr-featureset010/947c08e/logs/undercloud/home/zuul/.config/openstack/clouds.yaml.txt.gz | 09:50 |
| chandankumar | it worked on undercloud changes | 09:50 |
| marios | chandankumar: ack | 09:51 |
| *** ratailor__ has joined #oooq | 09:53 | |
| *** panda has quit IRC | 09:55 | |
| *** ratailor_ has quit IRC | 09:56 | |
| *** panda has joined #oooq | 10:00 | |
| *** holser_ has quit IRC | 10:23 | |
| *** holser_ has joined #oooq | 10:33 | |
| *** ykarel is now known as ykarel|meeting | 10:35 | |
| *** ykarel_ has joined #oooq | 10:59 | |
| *** jaosorior has joined #oooq | 10:59 | |
| *** ykarel_ has quit IRC | 11:00 | |
| *** ykarel has joined #oooq | 11:01 | |
| *** ykarel|meeting has quit IRC | 11:01 | |
| *** jaosorior has quit IRC | 11:02 | |
| *** jaosorior has joined #oooq | 11:03 | |
| *** udesale has quit IRC | 11:06 | |
| *** udesale has joined #oooq | 11:08 | |
| *** ykarel_ has joined #oooq | 11:09 | |
| *** ykarel has quit IRC | 11:12 | |
| *** ratailor__ has quit IRC | 11:18 | |
| chandankumar | sshnaidm: marios please have a look https://softwarefactory-project.io/r/#/c/15820/ when free, thanks :-) | 11:30 |
| marios | chandankumar: ok will do | 11:35 |
| zbr|ruck | who has some experience using dependencies feature on zuul job definition? | 11:38 |
| sshnaidm | zbr|ruck, we have tox jobs that use dependencies | 11:41 |
| sshnaidm | zbr|ruck, I think it's in tht repo | 11:41 |
| zbr|ruck | i will have a look but based on what i found so far I think we cannot do much about prios, look at 2nd page from https://docs.google.com/document/d/1u2q7E6SceTo5AxJf4vr9mZK08iAGOhQNJQXSYW1nIVM/edit# | 11:46 |
| zbr|ruck | maybe you have a better idea | 11:46 |
| zbr|ruck | i don't see how we can scale the dependencies feature when we have no clue about which jobs would exist at some point. | 11:46 |
| zbr|ruck | ideally each job should know its own priority-class instead of adding dependencies. | 11:47 |
| weshay | zbr|ruck you looking into the retry errors? | 11:47 |
| zbr|ruck | nope, was trying to find a solution of job triggering | 11:48 |
| zbr|ruck | weshay: looking now... | 11:50 |
| arxcruz | weshay, it seems ovb job on reproducer is more complicated than it seems to be | 11:51 |
| weshay | arxcruz ya.. we need to lean into it though.. make it better | 11:52 |
| chandankumar | weshay: RHEL-8 image is added to /var/lib/nodepool/rhel-guest-image-8.0-1854.x86_64.qcow2 we are updating nodepool config then it will available for ci usage | 11:56 |
| arxcruz | weshay, in the image that the job is supposed to run, do we have a clouds.yaml ? | 11:58 |
| arxcruz | weshay, we need a cloud (rdo) cloud account to run it | 11:59 |
| arxcruz | or find another solution | 11:59 |
| weshay | chandankumar++ | 11:59 |
| weshay | arxcruz can we use a secret in zuul and write out a clouds.yaml? | 12:00 |
| arxcruz | weshay, i never worked with secrets | 12:06 |
| arxcruz | sshnaidm, what do you think? | 12:06 |
| sshnaidm | arxcruz, about ..? | 12:07 |
| arxcruz | sshnaidm, reproducer and ovb | 12:07 |
| *** rfolco has joined #oooq | 12:12 | |
| chandankumar | can someone please look at downstream failure https://sf.hosted.upshift.rdu2.redhat.com/logs/58/174458/1/check/tripleo-ci-rhel-8-standalone-rhos-15/acbbca7/logs/undercloud/home/zuul/standalone_deploy.log.txt.gz#_2019-06-25_11_57_22 | 12:12 |
| weshay | sshnaidm using a secret in zuul jobs to store the password requied to build a clouds.yaml file for an ovb ci test | 12:13 |
| chandankumar | nerror pulling image \"192.168.24.1:8787/rhosp15/openstack-cinder-volume:20190621.1-updated-20190625074124 | 12:13 |
| weshay | w/ the reproducer | 12:13 |
| sshnaidm | weshay, it's not that simple with reproducer | 12:14 |
| weshay | is it at all possible? | 12:14 |
| sshnaidm | weshay, cloud secrets should be in zuul.yaml actually, below the job definition | 12:14 |
| sshnaidm | weshay, but firstly need to find a tenant to run there ovb job at all | 12:14 |
| arxcruz | sshnaidm, but we can't put the cloud_scrects in zuul.yaml because it has username and password right ? | 12:16 |
| arxcruz | {{ lookup('file', '{{ clouds_yaml_path }}') | regex_search('password: (.+)') }} | 12:16 |
| arxcruz | for example | 12:16 |
| weshay | sshnaidm arxcruz I'm in f.. mtgs all morning.. but sshnaidm we have a 1-1 later today where we can chat about it | 12:16 |
| sshnaidm | arxcruz, ya, we can put it actually, but need to hide it there in some way.. | 12:17 |
| sshnaidm | arxcruz, firstly need to find a tenant for that, we can't run it on openstack-nodepool tenant, it's not a good idea I suppose | 12:17 |
| arxcruz | sshnaidm, rdocloud or vexxhost ? | 12:18 |
| sshnaidm | arxcruz, then need to create secrets with this tenant, create some role with propagation of this secrets to job - in trusted repo only, then use them in job for writing in zuul.yaml and hiding this file even in debug, erasing it in some of roles. | 12:19 |
| zbr|ruck | weshay: infra is working on RETRY_LIMIT | 12:19 |
| sshnaidm | arxcruz, it's the smallest problem. | 12:19 |
| weshay | zbr|ruck any insight, or can you help? | 12:20 |
| sshnaidm | arxcruz, look on *openstack* job in reproducer jobs for some inspiration, quique did there auth with testing tenant | 12:20 |
| zbr|ruck | zuul config is messed and nope, i cannot help, neither the access or knowledge. but I am following the thread, hoping to learn about it. | 12:21 |
| zbr|ruck | as few others stated the finger message is probably the middle one from zuul. very opaque. | 12:21 |
| chandankumar | zbr|ruck: any help you need from prod chain team regarding zuul they will help you | 12:21 |
| chandankumar | ? | 12:22 |
| zbr|ruck | nope, osp-infra is working on it, is unrelated to us, see https://zuul.openstack.org/config-errors | 12:22 |
| weshay | marios https://review.opendev.org/#/c/667230/ need reviews | 12:23 |
| weshay | sshnaidm ^ | 12:23 |
| chandankumar | weshay: ah, thanks :-) | 12:23 |
| chandankumar | sorry | 12:23 |
| chandankumar | zbr|ruck: thanks :-) | 12:23 |
| sshnaidm | zbr|ruck, you looked for deps in zuul: https://review.opendev.org/#/c/667230/3/zuul.d/layout.yaml | 12:23 |
| zbr|ruck | sshnaidm: thanks! that is really cool, we can use globs for them. this should allow us to come up with a way to filter them. not as cool as regex but clearly better than no pattern. | 12:25 |
| marios | weshay: ack# | 12:25 |
| *** aakarsh has quit IRC | 12:29 | |
| *** rlandy has joined #oooq | 12:29 | |
| zbr|ruck | sshnaidm: i am looking at zuul code and I am afraid that there is no support for globs in "dependencies" | 12:37 |
| sshnaidm | zbr|ruck, well, good opportunity to contribute :) | 12:39 |
| zbr|ruck | sshnaidm: yeah, only if i would have time for that. but it is a feature I would like to add. | 12:40 |
| zbr|ruck | sshnaidm: also added https://storyboard.openstack.org/#!/story/2005951 to allow use of tags for dependencies. | 12:46 |
| rlandy | weshay: fyi ... https://code.engineering.redhat.com/gerrit/#/c/174574/ same as we did for needs_images. I have no idea why :( | 12:47 |
| rlandy | sshnaidm: mirror working ok for you? | 12:47 |
| rlandy | I merged a change to get bm working again | 12:48 |
| *** ykarel_ is now known as ykarel|away | 12:48 | |
| zbr|ruck | sshnaidm: i wonder how people are able to use storyboard, to be is seems like blackhole, almost impossible to search for simple keyword in a specific project. | 12:50 |
| chandankumar | rlandy: needs +w on this https://code.engineering.redhat.com/gerrit/#/c/174565/ | 12:51 |
| chandankumar | thanks:-) | 12:51 |
| sshnaidm | rlandy, didn't try it today | 12:51 |
| rlandy | sshnaidm: k- let me know if you hot any issues | 12:52 |
| chandankumar | rlandy: rhos-15 standalone is broken while pulling container | 12:52 |
| rlandy | chandankumar: just fixed it ... https://code.engineering.redhat.com/gerrit/#/c/174574/ | 12:53 |
| rlandy | chandankumar: and merged your change | 12:53 |
| rlandy | although zbr|ruck would not be happy about no linters :) | 12:53 |
| rlandy | in this case - no difference | 12:53 |
| chandankumar | rlandy: thanks :-) | 12:53 |
| arxcruz | sshnaidm, the openstack job in reproducer it's a dryrun | 12:53 |
| rlandy | chandankumar: pls try rhos-15 again and see if this fix works for you | 12:53 |
| *** skramaja has quit IRC | 12:54 | |
| chandankumar | weshay: rlandy the good thing is that donstream tempest project is gatted here https://code.engineering.redhat.com/gerrit/#/c/174458/2 | 12:54 |
| chandankumar | mkopec is complaining as it is failing on patch backport | 12:54 |
| * tosky sighs | 12:57 | |
| chandankumar | s/complaing/checking | 12:58 |
| chandankumar | tosky: we have removed the tox linters jobs from downstream | 12:58 |
| tosky | chandankumar: the sigh is not about the topic itself, but about the place where it's being discussed | 12:59 |
| arxcruz | weshay, sshnaidm can you point me one more time to the reproducer documentation? I lost my bookmarks | 13:00 |
| weshay | arxcruz it's in each job you joker | 13:00 |
| weshay | reproduer-readme | 13:00 |
| weshay | dang.. I need a new keyboard | 13:00 |
| weshay | 98 days until laptop refresh | 13:01 |
| *** ykarel|away has quit IRC | 13:01 | |
| arxcruz | weshay, http://logs.rdoproject.org/26/664926/6/openstack-check/tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset001/5fa75c1/logs/reproducer-quickstart/ | 13:01 |
| arxcruz | i don't see it here for example | 13:01 |
| arxcruz | weshay, 30 days for me :) | 13:01 |
| marios | http://logs.rdoproject.org/26/664926/6/openstack-check/tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset001/5fa75c1/logs/README-reproducer.html | 13:04 |
| marios | arxcruz: ^ one level up | 13:04 |
| arxcruz | marios, thanks, shit i would never check that :) | 13:05 |
| marios | arxcruz: yeah maybe we should move it or link it there or something | 13:05 |
| *** aakarsh has joined #oooq | 13:05 | |
| weshay | arxcruz where can we put it where you would? | 13:08 |
| weshay | not getting negative, just looking to improve | 13:10 |
| marios | weshay: in there http://logs.rdoproject.org/26/664926/6/openstack-check/tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset001/5fa75c1/logs/reproducer-quickstart/ | 13:10 |
| weshay | k.. can be in both.. it's just a template | 13:10 |
| arxcruz | weshay, yup, inside the reproducer-quickstart | 13:10 |
| weshay | k | 13:10 |
| weshay | easy change | 13:10 |
| arxcruz | weshay, usually, or me, is like, i go directly to undercloud directory | 13:11 |
| arxcruz | but that's me, because it's where tempest stuff is | 13:11 |
| weshay | chandankumar fyi https://review.opendev.org/#/c/666607/ | 13:15 |
| weshay | hrm.. the tox tests passed for me locally | 13:15 |
| chandankumar | weshay: https://softwarefactory-project.io/r/#/c/15820/ | 13:16 |
| *** ratailor has joined #oooq | 13:21 | |
| zbr|ruck | @oooq: due to some recent changes in zuul base jobs, you may discover some missing packages. If so, be sure that you add them to bindeps.txt file to fix it. | 13:24 |
| zbr|ruck | mainly that is a sideeffect of dropping bindep fallback list. no bindep means nothing, used to mean a list of implicit stuff. | 13:25 |
| rfolco | tripleo ci mtg starts in 5 min at https://bluejeans.com/5878458097, please add your agenda items to https://etherpad.openstack.org/p/tripleo-ci-squad-meeting @L57 | 13:25 |
| rfolco | marios, sshnaidm, weshay, panda, rlandy, arxcruz, rfolco, chkumar, zbr, kopecmartin | 13:25 |
| marios | thanks rfolco | 13:26 |
| rfolco | in 4... | 13:26 |
| rfolco | actually 3 | 13:27 |
| rfolco | ping community mtg: marios, sshnaidm, weshay, panda, rlandy, arxcruz, rfolco, chkumar, zbr, kopecmartin | 13:30 |
| *** Vorrtex has joined #oooq | 13:30 | |
| rfolco | boarding now on gate https://bluejeans.com/5878458097 | 13:31 |
| rfolco | weshay, are you able to join ? | 13:32 |
| arxcruz | marios, I'm so envy that you have a AC... it's freaking hot these days here in berlin | 13:34 |
| sshnaidm | zbr|ruck, do you want to explain about molecule job in current community call maybe? | 13:36 |
| chandankumar | zbr|ruck: please have a look at this issue https://logs.rdoproject.org/24/639324/35/openstack-check/tripleo-ci-centos-7-ovb-3ctlr_1comp_1supp-featureset039/9dc8a6b/logs/undercloud/home/zuul/overcloud_prep_images.log.txt.gz#_2019-06-25_12_38_16 | 13:37 |
| *** Goneri has joined #oooq | 13:40 | |
| *** saneax has quit IRC | 13:58 | |
| weshay | FYI... jim whitehurst update is happening now | 14:02 |
| *** Goneri has quit IRC | 14:03 | |
| *** ratailor has quit IRC | 14:04 | |
| *** ykarel has joined #oooq | 14:06 | |
| *** vinaykns has joined #oooq | 14:08 | |
| *** Goneri has joined #oooq | 14:16 | |
| rlandy | mirrors....ugh | 14:19 |
| *** dsneddon has quit IRC | 14:36 | |
| *** Goneri has quit IRC | 14:36 | |
| *** panda has quit IRC | 14:41 | |
| *** panda has joined #oooq | 14:43 | |
| *** ykarel is now known as ykarel|afk | 14:45 | |
| *** Goneri has joined #oooq | 14:50 | |
| *** udesale has quit IRC | 14:55 | |
| rlandy | downstream is getting very crowded | 14:57 |
| *** jpena|mtg is now known as jpena|off | 15:11 | |
| weshay | arxcruz hey.. join my blue for a sec.. chatting about the reproducer | 15:15 |
| *** ccamacho has quit IRC | 15:22 | |
| arxcruz | weshay, hey boss | 15:27 |
| arxcruz | sorry, i was out of computer | 15:27 |
| arxcruz | joining now | 15:27 |
| *** jaosorior has quit IRC | 15:38 | |
| chandankumar | sshnaidm: I am going to more patches to test overcloud tempest stuff in ci | 15:38 |
| *** jaosorior has joined #oooq | 15:38 | |
| chandankumar | going to port tripleo-ci-centos-7-containers-multinode to os_tempest | 15:39 |
| chandankumar | *put more | 15:39 |
| sshnaidm | chandankumar, cool | 15:46 |
| sshnaidm | chandankumar, so we can drop our validate-tempest role | 15:46 |
| *** holser__ has joined #oooq | 15:51 | |
| *** holser_ has quit IRC | 15:52 | |
| chandankumar | sshnaidm: yes, https://review.opendev.org/#/c/667390/ once it works and merges | 15:53 |
| chandankumar | we can port all jobs in a single sprint | 15:53 |
| chandankumar | for overcloud 6 patches are floating | 15:53 |
| *** hamzy has quit IRC | 15:54 | |
| sshnaidm | chandankumar++ | 15:54 |
| *** marios is now known as marios|out | 16:00 | |
| rlandy | is anybody's downstream job running? | 16:02 |
| chandankumar | rlandy: nope | 16:02 |
| chandankumar | rlandy: my few jobs were in queue state | 16:03 |
| *** udesale has joined #oooq | 16:03 | |
| rlandy | so are mine | 16:03 |
| rlandy | forever | 16:03 |
| * rlandy gives up to have lunch | 16:03 | |
| *** jfrancoa has quit IRC | 16:09 | |
| chandankumar | ok, /me logs out see ya tomorrow | 16:12 |
| *** chandankumar is now known as raukadah | 16:13 | |
| *** bogdando has quit IRC | 16:27 | |
| *** tesseract has quit IRC | 16:35 | |
| *** tesseract has joined #oooq | 16:36 | |
| *** holser__ has quit IRC | 16:40 | |
| *** panda has quit IRC | 16:46 | |
| *** panda has joined #oooq | 16:48 | |
| *** tesseract has quit IRC | 16:59 | |
| raukadah | rlandy: thanks for the rhos-15 fix, the standalone job is working now | 17:03 |
| rlandy | great | 17:04 |
| raukadah | rlandy: can we integrate it with rrcockpit? | 17:05 |
| *** marios|out has quit IRC | 17:05 | |
| rlandy | raukadah: I am working on that | 17:05 |
| rlandy | it's not a simple as I initially thought | 17:05 |
| raukadah | rlandy: great, thanks :-) | 17:05 |
| rlandy | we need to set up a new telegraf server | 17:05 |
| rlandy | in prcocess | 17:05 |
| *** udesale has quit IRC | 17:07 | |
| raukadah | rlandy: last week I was at rootconf event | 17:08 |
| raukadah | rlandy: there videos are up it might be helpful for us | 17:08 |
| raukadah | rlandy: sshnaidm zbr|ruck weshay https://www.youtube.com/watch?v=gTS4Agwwgbw&list=PL279M8GbNsesB7sxLO5zNbBHkOVvb7uvy | 17:09 |
| sshnaidm | raukadah, who is that? | 17:11 |
| *** raukadah is now known as chandankumar | 17:11 | |
| chandankumar | sshnaidm: it's me, just un unique username | 17:12 |
| chandankumar | https://rootconf.in/2019/ | 17:12 |
| *** hamzy has joined #oooq | 17:12 | |
| sshnaidm | niice :) | 17:13 |
| *** chandankumar is now known as raukadah | 17:13 | |
| weshay | rlandy have you been able to bring up a devel env for the cockpit? | 17:17 |
| *** jtomasek has quit IRC | 17:21 | |
| weshay | rlandy since your developement env would be behind the firewall, I think you should be able to prototype what you need w/ telegraph directly on your devel env | 17:21 |
| weshay | then we need to move that work to an internal box | 17:21 |
| weshay | rlandy for example.. changing the zuul url for the rdo jobs... to point to the internal zuul.. in your devel env.. | 17:22 |
| weshay | seeing if that loads the appropriate data.. | 17:22 |
| rlandy | weshay: not really - I was going to go back to it now | 17:34 |
| rlandy | since we can't get any jobs going on the downtream sf | 17:34 |
| weshay | rlandy need eyes? | 17:34 |
| weshay | mirrors still? | 17:34 |
| rlandy | node failures | 17:35 |
| rlandy | see comments above | 17:35 |
| weshay | bah.. upstream just hit that as well | 17:35 |
| * weshay reads | 17:35 | |
| weshay | rlandy ooff... internal sf looks like it's in bad shape | 17:38 |
| rlandy | weshay: I think I need to retire :( | 17:38 |
| rlandy | my whole life lives in this env now | 17:38 |
| rlandy | yesterday whole day run and games with mirrors | 17:38 |
| weshay | lolz | 17:38 |
| weshay | rlandy so backpedal off it.. you're supposed to be doing what ever you want atm | 17:39 |
| rlandy | whatever I want????? | 17:39 |
| weshay | what did I just trigger? | 17:39 |
| weshay | ya | 17:39 |
| rlandy | tripleo-ci-base-baremetal-microcloud-ha-2 tripleo-ci-base-baremetal-microcloud-ha-2 : NODE_FAILURE in 0s | 17:40 |
| rlandy | still | 17:40 |
| rlandy | weshay: yeah ok - going back to cockpit work | 17:40 |
| rlandy | at least that is it on my own system | 17:41 |
| weshay | rlandy so node_failures can be caused by the changes to the zuul playbooks | 17:41 |
| * weshay is assuming that is not the case here | 17:41 | |
| rlandy | or not enough nodes | 17:41 |
| rlandy | ugh - you right - maybe somebody changed something | 17:41 |
| * rlandy checks | 17:41 | |
| rlandy | tox runs ok though | 17:42 |
| * rlandy tries run again and watches | 17:42 | |
| weshay | https://code.engineering.redhat.com/gerrit/gitweb?p=openstack/tripleo-ci-internal-config.git;a=log and https://code.engineering.redhat.com/gerrit/gitweb?p=openstack/tripleo-ci-internal-jobs.git;a=log | 17:43 |
| weshay | rlandy I can pick up the node failures w/ nhicher | 17:43 |
| rlandy | hold on watching console | 17:43 |
| rlandy | https://sf.hosted.upshift.rdu2.redhat.com/zuul/t/tripleo-ci-internal/stream/b3b4da85c62544c28cb146d209706299?logfile=console.log | 17:44 |
| rlandy | running now | 17:44 |
| rlandy | I think we just overwhelmed the system before | 17:45 |
| *** ykarel|afk has quit IRC | 17:45 | |
| weshay | rlandy well.. I see git checkouts failing in the pre's | 17:49 |
| weshay | something is going on | 17:49 |
| weshay | https://sf.hosted.upshift.rdu2.redhat.com/logs/periodic-hourly/code.engineering.redhat.com/openstack/tripleo-ci-internal-jobs/master/check-tripleo-centos-7-rocky-containers-build/9125bf3/job-output.txt.gz#_2019-06-25_17_39_38_523352 | 17:49 |
| weshay | rlandy I think upshift is toast | 17:50 |
| rlandy | yeah - there was something going on | 17:51 |
| rlandy | I think we may be hard using it now | 17:51 |
| weshay | rlandy the cloud was down | 18:00 |
| weshay | lmiccini fyi ^ | 18:00 |
| zbr|ruck | weshay: check https://review.opendev.org/#/c/667423/3 -- is that in right direction? am I supposed to remove templates like this one and move their job into the new one? | 18:04 |
| zbr|ruck | doing such a shuffle could create a long list of interdependent changes. | 18:05 |
| weshay | zbr|ruck ya.. think so | 18:05 |
| zbr|ruck | for example that template is used by 3 ci repos. can I have recursive depends-on? | 18:06 |
| weshay | zbr|ruck I don't think so.. but try it.. create a test change in tq, that depends on tqe, | 18:07 |
| weshay | if you want to put a tripleo-ci change on top of that one to start.. top change -> tq -> tqe | 18:07 |
| weshay | and see what happens .. I think you'll be fine | 18:08 |
| weshay | know what I mean vern? | 18:08 |
| zbr|ruck | i think so, finding up soon. | 18:08 |
| *** holser_ has joined #oooq | 19:01 | |
| *** jtomasek has joined #oooq | 19:19 | |
| *** holser_ has quit IRC | 19:23 | |
| rfolco | weshay, sf vm is 30% slower to install the same 615 rpms. | 19:25 |
| zbr|ruck | weshay: so circular depends appears not to be working. what would you propose? just making the old template empty with TBD: remove? | 19:33 |
| *** Vorrtex has quit IRC | 19:38 | |
| *** tosky has quit IRC | 19:40 | |
| weshay | rfolco slower than rdo-cloud? | 19:58 |
| weshay | zbr|ruck /me looks | 19:58 |
| rfolco | weshay, slower than quickvm | 19:58 |
| weshay | rfolco ah.. that's fine | 19:59 |
| weshay | ok | 19:59 |
| weshay | rfolco but no failures... no crazy timeouts | 19:59 |
| weshay | rfolco w/ a proxy we should be fine | 19:59 |
| rfolco | weshay, closed out the task. | 19:59 |
| weshay | zbr|ruck you are putting up a full cirlcle | 20:01 |
| weshay | zbr|ruck does that ever work? | 20:01 |
| weshay | just have one straight shot... tq -> tqe -> tripleo-ci | 20:02 |
| * weshay doesn't see https://review.opendev.org/#/c/667423 in zuul atm | 20:03 | |
| weshay | weird | 20:03 |
| weshay | or https://review.opendev.org/#/c/667427/ | 20:03 |
| weshay | or 667423 | 20:03 |
| *** matbu has quit IRC | 20:13 | |
| weshay | rfolco thank you! | 20:26 |
| rfolco | weshay, testing mirror and logserver by tomorrow. | 20:27 |
| *** hamzy has quit IRC | 20:33 | |
| *** vkapalav has joined #oooq | 20:41 | |
| *** vinaykns has quit IRC | 20:43 | |
| *** Goneri has quit IRC | 20:49 | |
| *** aakarsh has quit IRC | 21:06 | |
| *** holser_ has joined #oooq | 21:07 | |
| rlandy | weshay: setting up my dev env for rrcockpit | 21:14 |
| rlandy | grafana | t=2019-06-25T21:14:16+0000 lvl=info msg="New state change" logger=alerting.resultHandler alertId=10 newState=ok prev state=alerting | 21:14 |
| rlandy | telegraf | 2019-06-25T21:14:20Z D! Output [influxdb] buffer fullness: 0 / 10000 metrics. | 21:14 |
| rlandy | ^^ grafana keeps reporting in red | 21:15 |
| rlandy | must be an issue | 21:15 |
| weshay | red | 21:15 |
| weshay | ? | 21:15 |
| rlandy | alerting | 21:15 |
| weshay | so there is some alerting.. but afaik it doesn't really work | 21:15 |
| weshay | you can see some sections w/ a red outline, is that what you are talking about? | 21:16 |
| rlandy | what port do I see the dashboard on? | 21:16 |
| weshay | we're supposed to get emails, but glad we dont | 21:16 |
| weshay | 8080 | 21:16 |
| weshay | I think | 21:16 |
| weshay | ya.. 8080 | 21:16 |
| rlandy | Unable to connect | 21:17 |
| weshay | rlandy blue? | 21:17 |
| weshay | tmate? | 21:18 |
| rlandy | blue, red, green, yellow | 21:18 |
| weshay | heh | 21:18 |
| rlandy | it keeps going | 21:18 |
| weshay | let's fix it | 21:18 |
| rlandy | yeah blueJEANS ok | 21:18 |
| *** jtomasek has quit IRC | 21:24 | |
| *** vkapalav has quit IRC | 21:30 | |
| *** aakarsh has joined #oooq | 21:44 | |
| *** holser_ has quit IRC | 21:48 | |
| *** aakarsh has quit IRC | 21:48 | |
| *** aakarsh has joined #oooq | 21:48 | |
| *** rlandy has quit IRC | 23:59 | |
Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!