Friday, 2019-11-22

« Thursday, 2019-11-21 Index Saturday, 2019-11-23 »
*** rlandy is now known as rlandy|bbl00:01
*** tosky has quit IRC00:56
*** openstackstatus has joined #oooq01:02
*** ChanServ sets mode: +v openstackstatus01:02
*** brault has joined #oooq01:56
*** rfolco has joined #oooq02:22
*** rfolco has quit IRC02:27
*** rlandy|bbl is now known as rlandy02:53
*** rlandy has quit IRC02:56
*** apetrich has quit IRC03:09
*** Trevor_V has quit IRC03:18
*** udesale has joined #oooq03:41
*** ysandeep has joined #oooq03:57
*** ysandeep is now known as ysandeep|bfast03:58
*** bhagyashris has joined #oooq04:02
*** dsneddon has quit IRC04:09
*** dsneddon has joined #oooq04:24
*** ratailor has joined #oooq04:30
*** dsneddon has quit IRC04:32
*** dsneddon has joined #oooq04:33
*** ysandeep|bfast is now known as ysandeep04:37
*** epoojad1 has joined #oooq04:48
*** dsneddon has quit IRC04:51
*** bhagyashris has quit IRC05:01
*** holser has joined #oooq05:05
*** bhagyashris has joined #oooq05:09
*** ratailor has quit IRC05:13
*** ratailor has joined #oooq05:18
*** sabedevops has joined #oooq05:23
*** dsneddon has joined #oooq05:26
*** surpatil has joined #oooq05:27
*** sabedevops has quit IRC05:29
*** soniya29 has joined #oooq05:40
*** holser has quit IRC05:55
*** rfolco has joined #oooq05:58
*** rfolco has quit IRC06:03
*** skramaja has joined #oooq06:05
*** zbr has joined #oooq06:06
*** zbr|ooo has quit IRC06:08
*** jtomasek has joined #oooq06:14
*** dsneddon has quit IRC06:19
*** marios|ruck has joined #oooq06:20
*** raukadah is now known as chkumar|rover06:37
chkumar|rovermarios|ruck: Hello06:46
chkumar|rovermarios|ruck: I have the reproducer up with the same failure, want to take a look?06:47
chkumar|roverswift ring_builder06:47
*** epoojad1 has quit IRC06:48
marios|ruckchkumar|rover: o/ morning - did you find anything interesting (maybe put on bug if you do). I can have a look - will you give me login info (I can check but in a bit). did you get anyone from swift (you said you ping some folks yesterday)06:50
chkumar|rovermarios|ruck: need public key06:51
marios|ruckchkumar|rover: https://github.com/marios.keys06:54
marios|ruckchkumar|rover: thanks06:54
chkumar|roverI have not checked with swift people, I will dig more into that06:54
marios|ruckchkumar|rover: stein promoted last night was almost red ... will dig at rocky today06:57
chkumar|roverok06:58
marios|ruckchkumar|rover: hmm strange it was green on testproject there https://review.rdoproject.org/r/#/c/23777/07:16
marios|ruckchkumar|rover: i think the mysql thing is red herring its also in the good log07:18
*** apetrich has joined #oooq07:22
*** holser has joined #oooq07:23
*** dsneddon has joined #oooq07:23
*** saneax has joined #oooq07:28
*** dsneddon has quit IRC07:28
chkumar|rovermarios|ruck: I think one of the bug is duplicate, but still not sure what is causing issue07:29
*** holser has quit IRC07:39
*** ysandeep has quit IRC07:46
chkumar|roverbrb after lunch07:47
marios|ruckchkumar|rover: it might not be just swift ringbuilder07:47
chkumar|rovermarios|ruck: mysql and swift proxy service is fine07:47
marios|ruckchkumar|rover: ack have good lunch (on your undercloud logs looks like more have the same eg            3470 2019-11-21 16:30:21 |         "2019-11-21 16:27:40,625 WARNING: 113350 -- Retrying running container: zaqar",            3249 2019-11-21 16:30:21 |         "2019-11-21 16:27:03,840 WARNING: 113348 -- Retrying running container: mistral",07:47
chkumar|rovermarios|ruck: cd /home/zuul/undercloud-ansible-2Yz_C5 and external_deploy_steps_tasks.yaml will i try re-running ansible config download playbook07:48
chkumar|rover?07:49
chkumar|roverbrb07:49
marios|ruckchkumar|rover: quite a few of those type=AVC msg=audit(1574353872.018:15602): avc:  denied  { read } for  pid=60509 comm="swift-container" name="4cc0e73e1ef327064588e0b6c7f535f5.db" dev="vda1" ino=6275895 scontext=system_u:system_r:container_t:s0:c432,c880 tcontext=system_u:object_r:swift_data_t:s0 tclass=file permissive=107:55
marios|ruckchkumar|rover: yeah we can re-run but lets poke first at logs a bit more07:56
marios|ruckchkumar|rover: so we know what to try07:56
*** ccamacho has joined #oooq08:09
*** ccamacho has quit IRC08:13
*** tosky has joined #oooq08:15
*** dsneddon has joined #oooq08:17
*** dsneddon has quit IRC08:23
*** ysandeep has joined #oooq08:25
*** ccamacho has joined #oooq08:27
*** tesseract has joined #oooq08:35
*** dsneddon has joined #oooq08:50
*** epoojad1 has joined #oooq09:03
*** chem has joined #oooq09:10
*** jfrancoa has joined #oooq09:19
*** sshnaidm is now known as sshnaidm|off09:21
*** jfrancoa has quit IRC09:24
*** jfrancoa has joined #oooq09:25
pandahot fresh reviews ! https://review.rdoproject.org/r/23773  https://review.rdoproject.org/r/2371709:28
pandaspoiler alert: they may take a while to review ...09:31
chkumar|rovermarios|ruck: http://zuul.openstack.org/builds?job_name=tripleo-ci-centos-7-containerized-undercloud-upgrades master is also broken now09:43
chkumar|rovermarios|ruck: "tripleo_ironic_pxe_tftp_healthcheck.service loaded failed failed ironic_pxe_tftp healthcheck"09:43
marios|ruckchkumar|rover: but is it same thing? for master we don't need to worry so much09:43
marios|ruckchkumar|rover: i mean for upgrade jobs on master09:43
marios|ruckchkumar|rover: so its weird that the testproject passed on fs1 stein (I think thats why we had the stein promotion). i saw the mysql errors on your undercluod, but i don't find them in the bugs logs09:44
*** derekh has joined #oooq09:44
marios|ruckchkumar|rover: i think you need to reach out to deployment team to dig09:44
marios|ruckchkumar|rover: i can't find something on your test box09:44
chkumar|rovermarios|ruck: I was trying to find a passed logs to compare the undercloud upgrade failure there I got the master failure09:45
marios|ruckchkumar|rover: i got excited when i found that         https://review.opendev.org/#/c/677237/ from 3 days ago but i can't see any nits there for example https://review.opendev.org/#/c/677237/29/deployment/swift/swift-proxy-container-puppet.yaml https://review.opendev.org/#/c/677237/29/deployment/swift/swift-storage-container-puppet.yaml09:45
marios|ruckchkumar|rover: ack yeah its always good to compare to good log cos some of the errors are misleading/also in good logs09:46
*** rascasoft has joined #oooq09:57
chkumar|rovermarios|ruck: Now I am totally confused, I was looking at passed job http://zuul.openstack.org/build/c25e5f823f31447881f7740c8999bddb and in https://7ac7dfcd0f6bacb47757-d2fc91d4dbb392695dd07e322bd6fba0.ssl.cf1.rackcdn.com/694773/3/check/tripleo-ci-centos-7-containerized-undercloud-upgrades/c25e5f8/logs/undercloud/home/zuul/undercloud_upgrade.log.txt.gz error regarding swift ring builder is there but uprgade10:05
chkumar|roverpassed10:05
chkumar|rover2019-11-21 17:29:44 |         "2019-11-21 17:29:42,169 ERROR: 126178 -- ERROR configuring swift_ringbuilder"10:05
marios|ruckchkumar|rover: where is that log from ... so it passes in check but not periodic?10:07
chkumar|rovermarios|ruck: I have grabbed the log from master check job and comparing with failed train job10:09
marios|ruckchkumar|rover: ah ok10:10
*** dtantsur|afk is now known as dtantsur10:22
*** ratailor has quit IRC10:49
*** soniya29 has quit IRC10:57
chkumar|rovermarios|ruck: panda fresh review little hotter https://review.opendev.org/#/c/694720/510:59
*** soniya29 has joined #oooq11:05
marios|ruckchkumar|rover: ack will check. fs1 stein now green on latest run :/ https://review.rdoproject.org/zuul/builds?job_name=periodic-tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset001-stein11:05
marios|ruckchkumar|rover: master promoted, train missed promotion only for fs10 so trying that https://review.rdoproject.org/r/#/c/23784/11:06
chkumar|rovermarios|ruck: ah, awesome11:06
chkumar|rovermarios|ruck: I wanted to chat about fs030 and standalone scenario7 job11:06
marios|ruckchkumar|rover: sure gimme few mins? we can get on bluej if you like? does half past work for you?11:07
marios|ruckchkumar|rover: like 23 mins11:07
chkumar|rovermarios|ruck: yes sounds good11:07
*** dsneddon has quit IRC11:08
*** yolanda__ has joined #oooq11:08
*** yolanda has quit IRC11:09
*** udesale has quit IRC11:10
pandachkumar|rover is there any way to test that change ?11:12
chkumar|roverpanda: https://review.rdoproject.org/r/#/c/23737/11:13
chkumar|roverpanda: I need one more help https://opendev.org/openstack/tripleo-quickstart-extras/src/branch/master/playbooks/tempest.yml#L6411:15
chkumar|roveri need to set undercloud: true in the given expression11:15
chkumar|roveri tried to through fs027 but it is not working https://review.opendev.org/#/c/694719/6/config/general_config/featureset027.yml@14111:16
pandachkumar|rover: ?11:16
chkumar|roverbasically i need to pull the skiplist of undercloud only11:17
chkumar|roverdo I need to add a seperate condition for undercloud?11:17
pandachkumar|rover: so you have a list of dictionaries, one of the keys in all elements is undercloud, and you want to select only the element that have unercloud key set to true ?11:18
chkumar|roverpanda: yes11:19
pandachkumar|rover: is the list in json ?11:19
chkumar|roverpanda: yes it is a list11:21
chkumar|roverpanda: v11:21
chkumar|roverpanda: https://2a14dbc9d75a16670d0e-9aa48c0c46296f03f9068df66e0862d4.ssl.cf2.rackcdn.com/694719/5/check/tripleo-ci-centos-7-undercloud-containers/a95a98e/logs/ara_oooq/result/bc07d023-1bf7-4069-9f14-02fef4b6aaa8/11:21
pandachkumar|rover: you need just the test: value ?11:25
pandachkumar|rover: or the whole list element ?11:26
chkumar|roverpanda: we need the whole list element having undercloud: true11:27
*** yolanda__ has quit IRC11:33
*** bhagyashris has quit IRC11:35
*** soniya29 has quit IRC11:36
pandachkumar|rover: I'm almost there11:41
pandachkumar|rover: last piece missing11:41
*** dsneddon has joined #oooq11:42
*** yolanda has joined #oooq11:46
chkumar|roverpanda: ok11:46
pandachkumar|rover: I have it, but there's a caveat11:47
*** epoojad1 has quit IRC11:48
pandachkumar|rover: and I have to understand a bit better how is known_failures passed to the task11:48
marios|ruckwtf the etherpad ate my edits :/11:49
pandamarios|ruck: it's lunchtime11:50
pandachkumar|rover: so if the known_failures is passed as json text, you need to convert if from json11:50
pandachkumar|rover: the you need to specify know_failures as key in this json.11:51
chkumar|roverpanda: https://opendev.org/openstack/openstack-ansible-os_tempest/src/branch/master/tasks/tempest_post_install.yml#L16711:52
pandachkumar|rover: then there's the caveat, you can't use undercloud=='true' for multiple reasons11:52
chkumar|roverpanda: list got passed,11:52
*** yolanda has quit IRC11:52
chkumar|roverthen key got selected therre11:52
pandachkumar|rover: I'm lost, wha'ts the relationship between known_failures and testpest_blacklist ?11:54
pandachkumar|rover: the important thing to undestand is if known_failures comes as json test or a ansible var to that task11:54
*** rfolco has joined #oooq11:55
chkumar|roverpanda: https://opendev.org/openstack/tripleo-quickstart-extras/src/branch/master/playbooks/tempest.yml#L6411:55
chkumar|rovertmepest_blacklist : known_failures11:55
pandachkumar|rover: ok, that comes after, where is known_failures defined ?11:55
chkumar|rover[https://opendev.org/openstack/tripleo-quickstart-extras/src/branch/master/roles/validate-tempest/vars/tempest_skip_master.yml#L711:56
chkumar|roverpanda: ^^11:56
pandachkumar|rover: ok, so it's already a ansible var, no need to convert11:59
pandachkumar|rover: ok, so the next step is the selection, and that's the caveat12:00
pandachkumar|rover: the boolean gets lost in some way, so you can't specify undercloud=='true' beacuse true is not text12:00
pandachkumar|rover: and you can't specify undercloud==true because the boolean true in the var is somewhat lost in the json_query filter12:01
chkumar|roverpanda: Do I need to add a another task to where we can specifu undercloud==true when tempest_cloud_name is undercloud?12:01
pandachkumar|rover: so you'll have to use just [?undercloud]12:01
chkumar|roverpanda: ok let me try that12:01
pandachkumar|rover: but you have to ensure taht if an "undercloud" key is present, then it should be true12:01
pandachkumar|rover: because the filter check for the presence of key undercloud12:02
pandachkumar|rover: also, i think [?undecloud==true] is dangerous anyway, because items taht don't have the undercloud key will match12:02
*** yolanda has joined #oooq12:05
chkumar|rovermarios|ruck: there is no train job for scenario-7, I need to add those also12:05
pandachkumar|rover: fs030 tested only for master, is that ok ?12:08
chkumar|roverpanda: it also runs on train,12:09
chkumar|roverpanda: I will add a job there to test it12:10
chkumar|rovergive me a min12:10
pandachkumar|rover: up to you, I can +212:10
chkumar|roverpanda: it will work anyway12:10
pandachkumar|rover: the ruck will fix it if it's broken12:10
chkumar|roverhehe12:11
chkumar|roverkopecmartin: agenda for https://hackmd.io/fIOKlEBHQfeTZjZmrUaEYQ?view tempest meeting12:12
*** soniya29 has joined #oooq12:14
marios|ruckchkumar|rover: ack12:14
marios|ruckpanda: images end up there https://images.rdoproject.org/centos7/rocky/rdo_trunk/ via fs2-upload job yes?12:29
*** ysandeep has quit IRC12:30
pandamarios|ruck: yes12:30
marios|ruckpanda: thanks12:30
marios|ruckpanda: tl;dr ... rocky promoted borked (not sure how) but it is trying to promote but fails cos ERROR    promoter ERROR: The promoted hash was not found: https://images.rdoproject.org/centos7/rocky/rdo_trunk/3e2f5658413cd31dd54933d0aed93da0068aed7e_fbb4de9612:31
marios|ruckpanda: luckily, that is still tripleo-ci-testing ^ https://trunk.rdoproject.org/centos7-rocky/tripleo-ci-testing/delorean.repo (cos it only runs wed/weekend) so hopefully i can run it with testproject and replace it indeed there is no image there with that https://images.rdoproject.org/centos7/rocky/rdo_trunk/ (bringing me back to i don't know how it happened)12:32
marios|ruckpanda: (i know, #cool story bro)12:32
pandamarios|ruck: I head last sprint that ruck/rover were trying to replace fs02-upload with a build-upload job ?12:32
marios|ruckpanda: yeah didn't happen yet12:33
marios|ruckpanda: i convinced them to wait until you are ruck12:33
pandamarios|ruck: great.12:33
pandamarios|ruck: any logs from fs02 that woulc explain what happened ?12:33
pandamarios|ruck: maybe we're out of space ?12:34
marios|ruckpanda: no in fact i don't see any fails there https://review.rdoproject.org/zuul/builds?job_name=periodic-tripleo-ci-centos-7-ovb-1ctlr_1comp-featureset002-rocky-upload12:34
marios|ruckpanda: well last one was a while ago anyway12:34
marios|ruckpanda: and this has been seen in logs for couple days now12:34
chkumar|roverpanda: http://logs.rdoproject.org/85/23785/1/check/molecule-delegated-promote-images/6a36099/job-output.txt TypeError: from_buffer() cannot return the address of the raw string within a str or unicode or bytearray object12:35
chkumar|roverpanda: knowne issue?12:35
pandachkumar|rover: yes, I have a patch for that taht si part of a larger patch, but maybe I have to put a separate patch for that now12:37
chkumar|roverpanda: cool, thanks :-)12:37
pandachkumar|rover: the other will take ages to merge ... :(12:37
*** skramaja has quit IRC12:37
chkumar|roverhmm12:37
marios|ruckchkumar|rover: i have the same there http://logs.rdoproject.org/37/23037/14/check/molecule-delegated-promote-images/18cbbf2/job-output.txt12:38
marios|ruckpanda: so we should make it non voting then12:38
marios|ruckpanda: chkumar|rover blocks that one https://review.rdoproject.org/r/#/c/23037/12:38
pandamarios|ruck: there's a delete script on the images server. If an image doesn't have a link associated, it's deleted12:39
pandamarios|ruck: so it's possibole that the promotion server took some time to promote, and in the meantime the image was deleted12:39
pandamarios|ruck: that's why you're not seeing failure in upload12:40
marios|ruckpanda: ack thanks that would explain12:40
marios|ruckpanda: maybe testproject will help us then lets see12:40
pandamarios|ruck: I'll push the paqtch for the delegated-promote-images, it's very small, and we can avoid putting in non-voting12:40
marios|ruckpanda: k12:40
pandamarios|ruck: chkumar|rover https://review.rdoproject.org/r/2378712:45
*** dsneddon has quit IRC12:45
chkumar|roverpanda: testing it12:46
*** skramaja has joined #oooq12:50
pandamarios|ruck: chkumar|rover: and that job anyway shoul have a file filterm so it should not run for any file that is not involved12:56
pandamarios|ruck: chkumar|rover was it blocking something unrelated ?12:56
chkumar|roverlet me check the job definition12:57
pandamarios|ruck: chkumar|rover if it was blocking on something related, the first solution should not be "put it non-voting" or you may merge broken code ...12:58
chkumar|roverpanda: https://github.com/rdo-infra/ci-config/blob/master/zuul.d/layout.yaml#L44 my change is promotion config changes sorunning there12:58
chkumar|roverpanda: https://review.rdoproject.org/r/#/c/23785/ it passed12:58
chkumar|roverweshay|ruck: kopecmartin tempest meeting time13:01
weshay|ruckhey13:01
*** dsneddon has joined #oooq13:03
*** rlandy has joined #oooq13:04
*** dsneddon has quit IRC13:08
*** epoojad1 has joined #oooq13:15
*** surpatil has quit IRC13:25
rlandymarios|ruck: you know when you just decide to take on a small task and you open pandora's box??? yep - the image build change was exactly that13:29
marios|ruckweshay|ruck: o/ re compute only build job at https://review.rdoproject.org/r/#/c/23655/4/zuul.d/build-containers.yaml where do you want to run that? should i add it with the rhel-8 build jobs like at https://github.com/rdo-infra/review.rdoproject.org-config/blob/master/zuul.d/tripleo.yaml or you don't want it running in ci/common/client etc ? do you want it running in check at all? or just merge it?13:30
*** soniya29 has quit IRC13:30
marios|ruckrlandy: ack, waiting on https://review.rdoproject.org/r/23786 hopefully will re-upload them13:30
marios|ruckrlandy: lucking the ci-testing hash is still same obviously the pipeline wont run again till weekend13:30
marios|ruckrlandy: oh you're referring to somethign else i think :D replacing fs2 with image build push13:31
rlandymarios|ruck: right13:31
marios|ruckrlandy: ignore the other thing (there is an image issue in rocky promoter missing images )13:31
rlandythe task I thought I would just finish off :)13:31
rlandynot me13:32
rlandymarios|ruck: we saw that happen a twice in our shift though13:32
weshay|ruckmarios|ruck, rfolco we want a new periodic pipeline that runs twice a day13:32
weshay|ruckcall the pipeline component-compute or something like that.. and drop the jobs in that13:32
*** dsneddon has joined #oooq13:33
rfolcoweshay|ruck, can we call generically like component-promotion ? I don't think we need a separate pipeline for each component13:33
rfolcoweshay|ruck, and what did you make change your mind about once vs twice a day?13:34
marios|ruckrfolco: weshay|ruck: ok then i think we can merge that https://review.rdoproject.org/r/#/c/23655/4/zuul.d/build-containers.yaml and then the pipeline/layout will come later (started there https://review.rdoproject.org/r/#/c/23712/2/zuul.d/upstream.yaml13:35
marios|ruckrfolco: weshay|ruck going to move it (back) to config repo then and call it periodic and it can merge there https://review.rdoproject.org/r/#/c/23655/4/zuul.d/build-containers.yaml13:39
weshay|ruckrfolco, at this time.. the design is an individual pipeline per component13:41
chkumar|roverrfolco: rlandy https://review.rdoproject.org/r/#/c/23787/ and then this https://review.rdoproject.org/r/#/c/23785/13:41
pandarfolco: need some eyes, especially on the 2nd https://review.rdoproject.org/r/23773 https://review.rdoproject.org/r/2371713:42
weshay|ruckrfolco, I need to see this run often atm for the poc13:42
weshay|ruckwe can throttle it down to once a day after the poc if needed13:42
*** bogdando has joined #oooq13:42
rfolcoweshay|ruck, ok, I am working on the changes to make the job read a custom release config file if its a component promotion job13:43
rfolcopanda, looking13:43
pandarfolco: also https://review.rdoproject.org/r/23787 https://review.rdoproject.org/r/23772 https://review.rdoproject.org/r/23775 https://review.rdoproject.org/r/23784 https://review.rdoproject.org/r/rfolcoooo https://review.rdoproject.org/r/1million https://review.rdoproject.org/r/theinternet13:43
rfolcopanda, ok, queued for 2030, thanks13:44
pandarfolco: https://review.rdoproject.org/r/2030 too then13:44
rlandychkumar|rover: https://review.rdoproject.org/r/#/c/23785/2/ci-scripts/dlrnapi_promoter/config/CentOS-7/master.ini is not going to impact your current promotions13:44
rlandycriteria are frozen13:44
chkumar|roverrlandy: hmm13:44
chkumar|roverrlandy: is it possible to patch manually then if no issue?13:45
rlandychkumar|rover: you need to get on to the promotion server and edit the criteria there13:45
chkumar|roverok13:45
rlandychkumar|rover: do you want me to take you through that?13:45
rlandyit's quick13:45
chkumar|roverrlandy: yup13:46
rlandysee pm13:46
chkumar|roverrlandy: where we update the ssh keys in config repo?13:46
rfolcopanda, can this file read a zuul var set in the job definition ? https://review.opendev.org/#/c/694836/8/config/release/tripleo-ci/CentOS-7/promotion-testing-hash-master.yml13:46
rlandychkumar|rover: it was in infra setup13:48
chkumar|roverrlandy: I will update there13:48
rlandybut for now - I'll try add ou13:48
rlandyyour keys13:48
pandarfolco: a yaml file can't read. Didn't go to read school13:49
rfolcohow this would work ? delorean_hash_label: &promotion-testing-tag "{{ dlrn_hash|default(dlrn_hash_tag) }}"13:49
pandarfolco: but in our jobs we are dumping the zuul inventory in a file, and then we load the file in our execution, so I think the yaml can reference a variable there13:50
rfolcopanda, ok will test it13:50
rfolcothx13:50
pandarfolco: I don't remember if we pass all the vars in zuul, or just a subset, check eventually.13:51
marios|ruckchkumar|rover: going afk in few mins fyi will be back soon (I'll renick)13:52
weshay|ruckrfolco, marios|ruck any questions about my comments re: the component pipeline?13:52
marios|ruckweshay|ruck: added comment there about 2/day fwiw https://review.rdoproject.org/r/#/c/23712/2/zuul.d/upstream.yaml13:53
rfolcoweshay|ruck, individual pipeline for each component, twice a day13:53
marios|ruckweshay|ruck: but i think rfolco will be working there this sprint more than me13:53
marios|ruckrfolco: feel free to abandon that https://review.rdoproject.org/r/#/c/23712/2/zuul.d/tripleo.yaml if you want to post your own or take it over13:54
rfolcoyes and I heard rlandy also willing to work on this13:54
rlandyrfolco: rlandy is willing13:54
rfolcorlandy, rlandy speaks in 3rd person13:54
marios|ruckrfolco: the rlandy is not amused13:55
rlandyrlandy is busy - sec13:55
marios|ruckchkumar|rover: ignore, not leaving change of plans (i have a flat tyre \o/  :( so cant be taxi)13:56
rlandychkumar|rover sabotages marios|ruck car so he doesn't leave?13:59
chkumar|roverhehe13:59
*** jaosorior has joined #oooq14:00
*** skramaja has quit IRC14:06
chkumar|roverrlandy: regarding new train jobs, do we need to add them first in rdo-jobs then in upstream or i do opposite?14:08
rlandychkumar|rover; depends how you want them parented14:08
chkumar|roverupstream to downstream14:08
rlandychkumar|rover: if all the definitions are in, you can activate them in any order14:09
rlandywhat is in the gating should be in periodic14:09
chkumar|roverworking on patche14:10
rlandychkumar|rover: first I would add all the definitions and make sure you are not missing jobs14:10
rlandythen add non-voting to check14:10
rlandyrun three days14:10
rlandyremove non-voting - add to gate14:10
marios|ruckrfolco: panda when you have time review that please https://review.rdoproject.org/r/#/c/23595/ its just the nova container buildjob definition14:17
rfolcomarios|ruck, ack, probably after lunch14:17
rfolcopanda, same for your request14:17
marios|ruckrfolco: no rush, i just updated it based on our discussions last couple days about keeping the build job without push in the pipeline14:17
rlandyrfolco: do you want to catch me up on the component pipeline stuff?14:18
rfolcorlandy, gimme a few and we'll sync14:19
rlandyack14:19
*** epoojad1 has quit IRC14:20
* marios|ruck biab14:34
*** marios|ruck is now known as marios|ruck|afk14:34
rfolcorlandy, I am ready, can you talk now?14:41
rlandyrfolco: yes14:41
weshay|ruckrlandy, rfolco bring me into that please14:42
rlandyweshay|ruck: sorry you can only join this meeting if you have a nick starting with r*14:42
rlandythe |ruck thing might be passable14:43
rfolcoweshay|ruck, rlandy https://meet.google.com/bqx-xwht-wky14:43
*** skramaja has joined #oooq14:52
*** ratailor has joined #oooq14:59
*** TrevorV has joined #oooq15:00
*** Goneri has joined #oooq15:17
*** marios|ruck|afk is now known as marios|ruck15:19
rlandyweshay|ruck: fyi - the infra guys want a stack clean up job on vexxhost like we run on rdocloud15:26
rlandyto do that, we would may want an ci-infra node on vexxhost15:27
rlandyor we could use the same node and store the vexx keywords15:27
rlandychkumar|rover: sshnaidm|off: ^^ fyi15:27
weshay|ruckrlandy, ya.. so vexx should be setup for us exactly as rdo cloud15:32
sshnaidm|offrlandy, it's one of techdebts that are waiting for proper time :)15:32
weshay|ruckwe'll want a ci-admin tenant15:32
weshay|ruckwe'll want a rhui15:32
weshay|ruckwe'll want a container registry15:32
*** ccamacho has quit IRC15:32
weshay|ruckI'll have to restart those conversations w/ Alan and crew15:33
rlandysshnaidm|off: weshay|ruck: fine - just want agreement that we want to use a specific node on vexx15:33
rlandyand not try contact it from rdocloud node15:33
rlandyit seems that are desperate for the clean scripts15:34
sshnaidm|offrlandy, well, not really15:34
sshnaidm|offrlandy, we don't need a node for that15:34
sshnaidm|offit will be an additional maintenance15:34
marios|ruckchkumar|rover: \o/ https://review.rdoproject.org/r/#/c/23786/ and rocky promoting too now we had master/stein/rocky today. missed train :/15:36
marios|ruckpanda: fyi it worked ^15:36
chkumar|rover\o/15:37
chkumar|roverawesome15:37
rlandysshnaidm|off: the alternative is to use our current node15:37
rlandyand just add a vexx run15:37
rlandyall we need are the tenant creds15:37
sshnaidm|offrlandy, weshay|ruck https://tree.taiga.io/project/tripleo-ci-board/task/51415:39
pandamarios|ruck: when's the next ?15:39
rlandyok15:39
marios|ruckpanda: you mean for rocky? weekend (we/weekend pipe)15:39
panda"marios|ruck: missed train :/ "15:40
marios|ruckpanda: yeah15:40
marios|ruckpanda: i got there in the end ;)15:40
pandamarios|ruck: by taxi ?15:40
marios|ruck:(15:41
sshnaidm|offweshay|ruck, rlandy and what we should do is https://tree.taiga.io/project/tripleo-ci-board/issue/119715:42
sshnaidm|offweshay|ruck, rlandy it will reduce significantly stacks leftovers if not remove them completely15:42
* weshay|ruck looks15:43
rlandysshnaidm|off: so I am in favor15:43
rlandyin the mean time, should we not just setup clean up to help them out with the poc stuff15:43
weshay|rucksshnaidm|off, so everything we need is merged? https://review.opendev.org/#/q/topic:cleanup-phase+(status:open+OR+status:merged)15:43
sshnaidm|offweshay|ruck, yeah, long time ago15:44
sshnaidm|offrlandy, just let's use the same scripts with vexx credentials for now15:44
rlandywell - how far off are we from implementing this?15:44
weshay|ruckok.. seems like a better path forward than what we have now15:44
rlandymaybe it's quick15:45
rlandyseems pretty easy now that I look at it15:45
rlandyk - let's try this15:46
*** ratailor has quit IRC15:47
*** saneax has quit IRC15:59
*** saneax has joined #oooq16:03
*** jbadiapa has joined #oooq16:03
marios|ruckpanda: can you revisit when you get some time please thanks https://review.rdoproject.org/r/#/c/23037/ cc rfolco please add to your reviews16:11
rfolcomarios|ruck, molecule-delegated-promote-imagesFAILURE in 3m 26s16:12
marios|ruckrfolco: yeah its in recheck it was fixed by another review (unrelated) I commented about that today there16:13
marios|ruckrfolco: "molecule-delegated-promote-images hopefully fixed by "Bump cryptography version in molecule delegated job" Change-Id: Iee018f7f74b875644d4777d8e17c7e9cb36fa6f9 https://review.rdoproject.org/r/#/c/23787/"16:13
rfolcomarios|ruck, ack16:13
marios|ruckrfolco: literally just reported green :)16:13
marios|ruckrfolco: ask me about the failing train promotion quick !16:14
rfolcohoes is the promotion train marios|ruck16:15
marios|ruckrfolco: hmm let me check16:15
marios|ruckrfolco: :( didn't work16:15
rfolcowe missed the train16:15
marios|rucknot you too panda was enough16:15
rfolcoI like silly puns16:16
marios|ruckrfolco: so remember how we wanted to be able to override the hash in the periodic jobs with testproject. why did we decide that was a bad idea again? it would be very helpful for chasing promotions16:17
marios|ruckrfolco: like i need to run a particular job (fs10) which i did and it passed, but not for the hash i wanted16:17
rfolcomarios|ruck, its not because it was a bad idea, its because the hash you used in your job was not the one reported16:17
rfolcomarios|ruck, also the var name was not just dlrn_hash, was something else16:18
marios|ruckrfolco: oh, it overrides in the end no matter what ... that rings a bell16:18
marios|ruckrfolco: but we *could* fix that?16:18
rfolcomarios|ruck, I had a patch for it, let me check16:18
marios|ruckrfolco: no you in fact merged it and we reverted16:18
rfolcomarios|ruck, I remember sagi -2 it16:18
rfolcoyes16:19
marios|ruckrfolco: well you merged somethign then we wanted to fix it and we decided in the end to revert instead something like that16:19
rfolcothats why, according to sagi's point, we should not do it, report an arbitrary hash16:20
rfolcothe hash should be hardcoded as it is16:20
marios|ruckrfolco: ack ok thanks was just thinking about it today and couldn't remember. ok hopefully we get a promotion on the weekend.16:20
rfolcomarios|ruck, may the luck be with you16:20
rfolcoif we get an interplanetary alignment, promotions happen16:21
marios|ruck:D16:21
pandamarios|ruck: small nit.16:23
pandamarios|ruck: well, small miss from me16:23
marios|ruckpanda: does that mean i will have to rewrite 99.4% of the patch16:23
pandamarios|ruck:  99.4% is optimistic. I changed my mind, I think we want to rewrite the promoter in ruby.16:24
marios|ruckhttps://i.kym-cdn.com/entries/icons/original/000/000/091/TrollFace.jpg  "just small inline nit otherwise lgtm"16:24
marios|rucklike that ? panda ^16:24
*** bogdando has quit IRC16:24
marios|ruckpanda: i just saw a nit too16:25
pandamarios|ruck:  the urrlib ?16:25
marios|ruckpanda: yea was that yours too?16:25
pandamarios|ruck: yep16:25
marios|ruckpanda: thanks done16:26
pandamarios|ruck: givve 5 minutes, checking tests16:26
marios|ruckpanda: ack :D thanks16:26
chkumar|roverrust is so cool, so why go for ruby16:28
pandachkumar|rover: i would have voted for LOLcode, frankly.16:28
pandamarios|ruck: aaaaa, I was almost going to -1 ... shame16:30
pandamarios|ruck: the two dlrn_hashes in the test seemed equal for .03 seconds16:30
marios|ruckpanda: yes i just randomly replaced some chars16:30
* chkumar|rover leaves now, unless panda kicks me again16:31
chkumar|rovermarios|ruck: happy weekend mate :-)16:31
*** chkumar|rover is now known as raukadah16:31
marios|ruckpanda: i think he is asking you to ^^^ i think he secretly likes it16:31
marios|ruckraukadah: have a good one !16:31
pandachkumar|rover: I hurt you because I love you.16:31
marios|rucklol16:31
pandarfolco: if you don't review my patches I'll love you even more.16:35
*** jfrancoa has quit IRC16:45
*** yolanda has quit IRC16:50
marios|ruckweshay|ruck: /me almost out need something before i do?17:01
weshay|ruckmarios|ruck, no sir.. have a good weekend17:01
marios|ruckweshay|ruck: thanks same to you17:02
*** marios|ruck is now known as marios|out17:04
*** jaosorior has quit IRC17:04
*** marios|out has quit IRC17:13
*** dtantsur is now known as dtantsur|afk17:21
*** tesseract has quit IRC17:35
*** jaosorior has joined #oooq17:44
*** rascasoft has quit IRC18:00
*** derekh has quit IRC18:00
*** skramaja has quit IRC18:02
rlandyweshay|ruck: so with some help from raukadah, this looks like it may work https://review.opendev.org/#/c/695591/18:19
rlandytesting it out next in testproject18:19
weshay|ruckhrm. I thought I saw the centos7 job passing earlier18:20
weshay|ruckoh.. this is the improvement18:20
weshay|ruckcool cool18:21
*** dsneddon has quit IRC18:23
*** irclogbot_0 has quit IRC18:27
rlandyweshay|ruck: yep - once it passes - will put dmitry on the review and let him comment18:29
*** irclogbot_1 has joined #oooq18:30
rlandy2019-11-22 16:57:32.975 | + set -o pipefail18:30
rlandy2019-11-22 16:57:32.975 | + chkconfig network on18:30
rlandy2019-11-22 16:57:32.976 | failed to glob pattern /etc/rc0.d/[SK][0-9][0-9]network: No such file or directory18:30
rlandystupid error will do us in18:30
*** bogdando has joined #oooq18:59
*** jaosorior has quit IRC19:08
*** saneax has quit IRC20:05
*** dsneddon has joined #oooq20:06
*** yolanda has joined #oooq20:10
*** dsneddon has quit IRC20:18
*** dsneddon has joined #oooq20:46
*** dsneddon has quit IRC20:48
weshay|ruckrlandy, no need for votes atm.. but fyi https://review.opendev.org/#/c/695761/20:51
*** rlandy has quit IRC21:18
*** rfolco has quit IRC21:58
*** Goneri has quit IRC22:02
*** dsneddon has joined #oooq22:06
*** kopecmartin is now known as kopecmartin|off22:18
*** bogdando has quit IRC22:20
*** dsneddon has quit IRC22:31
*** dsneddon has joined #oooq22:56
*** dsneddon has quit IRC23:05
*** dsneddon has joined #oooq23:28
*** TrevorV has quit IRC23:30
*** jbadiapa has quit IRC23:42
*** tosky has quit IRC23:45
*** jtomasek has quit IRC23:49
*** dsneddon has quit IRC23:58
« Thursday, 2019-11-21 Index Saturday, 2019-11-23 »

Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!