| mwhahaha | Wonder if the 401 is getting cached or soemthing | 00:05 |
|---|---|---|
| openstackgerrit | Kevin Carter (cloudnull) proposed openstack/tripleo-common master: Log exceptions when checking status https://review.opendev.org/674919 | 00:21 |
| cloudnull | they're happening at different times, and we can now see the auth keys are changed. so idk why it'd be getting a 401 | 00:22 |
| cloudnull | maybe there is something up with the rsyslog image ? | 00:22 |
| *** bfournie has joined #tripleo | 00:34 | |
| *** altlogbot_0 has quit IRC | 00:46 | |
| *** altlogbot_2 has joined #tripleo | 00:48 | |
| openstackgerrit | Merged openstack/puppet-tripleo master: Make puppet-tripleo compatible with latest puppet-rabbitmq https://review.opendev.org/677173 | 00:57 |
| openstackgerrit | Kevin Carter (cloudnull) proposed openstack/tripleo-heat-templates master: Convert firewall rules to use TripleO-Ansible https://review.opendev.org/677237 | 01:04 |
| *** bhagyashris has joined #tripleo | 01:11 | |
| *** Vorrtex has joined #tripleo | 01:23 | |
| *** ekultails has quit IRC | 01:35 | |
| *** xarlos has quit IRC | 01:49 | |
| *** d0ugal has quit IRC | 01:52 | |
| *** mschuppert has quit IRC | 01:54 | |
| openstackgerrit | Takashi Kajinami proposed openstack/tripleo-heat-templates master: Require service token role for Barbican api https://review.opendev.org/677326 | 02:05 |
| *** d0ugal has joined #tripleo | 02:06 | |
| *** apetrich has quit IRC | 02:10 | |
| openstackgerrit | Takashi Kajinami proposed openstack/tripleo-heat-templates master: Specify service_token_roles in keystone authtoken https://review.opendev.org/677328 | 02:13 |
| tonyb | Is there a 'trivial' way to go from an environment file (eg environments/services/ironic.yml) to map that back to the container images needed for that? | 02:44 |
| tonyb | Also it'd be good to get the same information for Roles -> Images | 02:45 |
| *** spsurya has joined #tripleo | 03:04 | |
| mcarden | tonyb: You just gotta do whatever `openstack overcloud container image prepare` is doing under the covers. :) | 03:25 |
| *** gkadam has joined #tripleo | 03:33 | |
| *** gkadam has quit IRC | 03:40 | |
| *** ramishra has joined #tripleo | 03:41 | |
| *** psachin has joined #tripleo | 03:44 | |
| *** skramaja has joined #tripleo | 03:53 | |
| *** rlandy|ruck|bbl is now known as rlandy|ruck | 03:56 | |
| *** ramishra has quit IRC | 04:00 | |
| *** udesale has joined #tripleo | 04:02 | |
| *** rlandy|ruck has quit IRC | 04:02 | |
| *** ykarel has joined #tripleo | 04:07 | |
| tonyb | mcarden: Sure but I'm not sure that's entirely what I want ... but I shall investigate | 04:08 |
| *** tkajinam has quit IRC | 04:09 | |
| mcarden | Well, it does spit out container image names to the file named at --output-env-file based on service env files you've handed it, so it's at least doing *some* of what you want. | 04:10 |
| ykarel | tonyb, you can check https://github.com/openstack/tripleo-common/blob/master/container-images/overcloud_containers.yaml.j2 for service vs image mapping | 04:15 |
| ykarel | tonyb, can u please check http://lists.openstack.org/pipermail/openstack-discuss/2019-August/008486.html and confirm nothing else apart from requesting ocata-eol tag in openstack/releases is needed for moving tripleo projects to ocata-eol | 04:16 |
| *** tkajinam has joined #tripleo | 04:16 | |
| *** tkajinam has quit IRC | 04:17 | |
| *** tkajinam has joined #tripleo | 04:18 | |
| openstackgerrit | Sagi Shnaidman proposed openstack/tripleo-quickstart-extras master: Add overcloud deploy parameter to set CinderLVMLoopDeviceSize https://review.opendev.org/677227 | 04:31 |
| *** ratailor has joined #tripleo | 04:46 | |
| *** ykarel is now known as ykarel|afk | 04:49 | |
| *** ykarel|afk has quit IRC | 04:53 | |
| *** Vorrtex has quit IRC | 05:04 | |
| *** janki has joined #tripleo | 05:10 | |
| Tengu | hello there! | 05:14 |
| *** raukadah is now known as chkumar|rover | 05:15 | |
| openstackgerrit | Merged openstack/puppet-tripleo stable/queens: [queens only] Add registry login support https://review.opendev.org/674955 | 05:17 |
| openstackgerrit | Merged openstack/tripleo-heat-templates stable/queens: [Queens] Enable container auth support https://review.opendev.org/674956 | 05:17 |
| openstackgerrit | Merged openstack/tripleo-common-tempest-plugin master: Update to stestr https://review.opendev.org/677040 | 05:17 |
| openstackgerrit | Merged openstack/tripleo-quickstart-extras master: Add basic minion validation https://review.opendev.org/675201 | 05:17 |
| openstackgerrit | Merged openstack/tripleo-image-elements stable/stein: Fix line comparisons https://review.opendev.org/675915 | 05:17 |
| openstackgerrit | Merged openstack/tripleo-image-elements stable/rocky: Fix line comparisons https://review.opendev.org/675916 | 05:17 |
| openstackgerrit | Merged openstack/tripleo-image-elements stable/queens: Fix line comparisons https://review.opendev.org/675917 | 05:17 |
| *** ykarel|afk has joined #tripleo | 05:19 | |
| *** ykarel|afk is now known as ykarel | 05:19 | |
| *** cfontain_ has joined #tripleo | 05:27 | |
| *** shyamb has joined #tripleo | 05:27 | |
| *** cfontain has quit IRC | 05:30 | |
| *** cfontain_ has quit IRC | 05:32 | |
| *** jaosorior has joined #tripleo | 05:32 | |
| *** sanjayu_ has joined #tripleo | 05:32 | |
| *** ykarel is now known as ykarel|meeting | 05:33 | |
| *** ratailor_ has joined #tripleo | 05:35 | |
| *** kopecmartin|off is now known as kopecmartin | 05:37 | |
| *** ykarel has joined #tripleo | 05:37 | |
| *** ratailor__ has joined #tripleo | 05:38 | |
| *** ykarel|meeting has quit IRC | 05:38 | |
| *** ratailor has quit IRC | 05:38 | |
| *** ratailor_ has quit IRC | 05:40 | |
| *** yprokule has joined #tripleo | 05:40 | |
| *** ramishra has joined #tripleo | 05:41 | |
| *** waleedm has joined #tripleo | 05:44 | |
| openstackgerrit | Cédric Jeanneret (Tengu) proposed openstack/tripleo-ansible master: Creates tripleo-validations-package role https://review.opendev.org/677229 | 05:44 |
| *** yprokule has quit IRC | 05:45 | |
| *** yprokule has joined #tripleo | 05:47 | |
| *** noama has joined #tripleo | 05:51 | |
| *** shyamb has quit IRC | 05:57 | |
| *** shyamb has joined #tripleo | 06:05 | |
| openstackgerrit | Francesco Pantano proposed openstack/puppet-tripleo master: Add certmonger-grafana-refresh script https://review.opendev.org/676395 | 06:19 |
| openstackgerrit | Chandan Kumar (raukadah) proposed openstack/tripleo-quickstart master: Use config_drive for network basicops tests https://review.opendev.org/676439 | 06:22 |
| *** surpatil has joined #tripleo | 06:25 | |
| *** ykarel is now known as ykarel|meeting | 06:26 | |
| *** udesale has quit IRC | 06:33 | |
| *** udesale has joined #tripleo | 06:34 | |
| *** rcernin has quit IRC | 06:35 | |
| *** mschuppert has joined #tripleo | 06:36 | |
| *** florianf has joined #tripleo | 06:38 | |
| *** avivgta has joined #tripleo | 06:50 | |
| *** egonzalez has joined #tripleo | 06:51 | |
| openstackgerrit | Michele Baldessari proposed openstack/puppet-tripleo stable/stein: Make the rabbitmq-ready exec more stringent https://review.opendev.org/677404 | 06:52 |
| openstackgerrit | Michele Baldessari proposed openstack/puppet-tripleo stable/rocky: Make the rabbitmq-ready exec more stringent https://review.opendev.org/677405 | 06:52 |
| openstackgerrit | Michele Baldessari proposed openstack/puppet-tripleo stable/queens: Make the rabbitmq-ready exec more stringent https://review.opendev.org/677406 | 06:53 |
| *** cylopez has joined #tripleo | 07:01 | |
| *** udesale has quit IRC | 07:01 | |
| *** udesale has joined #tripleo | 07:02 | |
| openstackgerrit | Zijian Guo proposed openstack/os-net-config master: Make class vlan support name https://review.opendev.org/677409 | 07:04 |
| openstackgerrit | Zijian Guo proposed openstack/os-net-config master: Make class vlan support name https://review.opendev.org/677409 | 07:06 |
| openstackgerrit | Ian Wienand proposed openstack/diskimage-builder master: simple-init: add configurable RA timeout with network-manager https://review.opendev.org/677410 | 07:07 |
| openstackgerrit | Zijian Guo proposed openstack/os-net-config master: Make class Vlan support name https://review.opendev.org/677409 | 07:07 |
| openstackgerrit | Zijian Guo proposed openstack/os-net-config master: Make class Vlan support name https://review.opendev.org/677409 | 07:10 |
| openstackgerrit | Carlos Goncalves proposed openstack/tripleo-heat-templates stable/rocky: Use docker_container Ansible module https://review.opendev.org/677411 | 07:19 |
| *** amoralej|off is now known as amoralej | 07:19 | |
| openstackgerrit | Michele Baldessari proposed openstack/tripleo-heat-templates master: HA: reorder init_bundle and restart_bundle for improved updates https://review.opendev.org/675993 | 07:23 |
| openstackgerrit | Carlos Goncalves proposed openstack/tripleo-heat-templates stable/queens: Use docker_container Ansible module https://review.opendev.org/677413 | 07:23 |
| *** psachin has quit IRC | 07:25 | |
| openstackgerrit | Natal Ngétal proposed openstack/python-tripleoclient master: Use python docstring. https://review.opendev.org/677418 | 07:29 |
| gchamoul | Tengu: Hi there! when you have a sec, could you please check this out? https://review.opendev.org/#/q/topic:bp/validation-framework-infra-backport+(status:open) | 07:29 |
| Tengu | gchamoul: heya! on my way :). | 07:30 |
| Tengu | gchamoul: aren't you on PTO? :D | 07:30 |
| gchamoul | Tengu: this afternoon after our meeting! | 07:31 |
| Tengu | gchamoul: cool! | 07:31 |
| Tengu | gchamoul: hmmm.... we're still supposed to get 2 reviwers even for backports right? | 07:31 |
| gchamoul | Tengu: yes, but it's tripleo-validations | 07:32 |
| gchamoul | and it's mainly molecule tests | 07:32 |
| Tengu | are you pushing me toward making +CR +A? ^^ | 07:32 |
| *** pierreprinetti has joined #tripleo | 07:33 | |
| gchamoul | Tengu: I did it yesterday on master for your molecule patches! | 07:33 |
| Tengu | XD | 07:33 |
| Tengu | fine then | 07:33 |
| gchamoul | Tengu: it's all greenish | 07:33 |
| Tengu | just waiting for the last one, zuul wasn't happy | 07:33 |
| *** rascasoft has quit IRC | 07:35 | |
| gchamoul | ack | 07:35 |
| *** xek has joined #tripleo | 07:39 | |
| *** bhagyashris has quit IRC | 07:39 | |
| *** rascasoft has joined #tripleo | 07:39 | |
| *** shyamb has quit IRC | 07:39 | |
| *** jpena|off is now known as jpena | 07:41 | |
| *** nawar has joined #tripleo | 07:47 | |
| nawar | hi | 07:47 |
| nawar | is this a known bug | 07:49 |
| nawar | File \"/usr/lib64/python2.7/multiprocessing/pool.py\", line 554, in get", | 07:49 |
| nawar | " raise self._value", | 07:49 |
| nawar | "OSError: [Errno 2] No such file or directory" | 07:49 |
| *** ykarel has joined #tripleo | 07:50 | |
| *** ratailor has joined #tripleo | 07:50 | |
| *** ratailor__ has quit IRC | 07:50 | |
| *** ykarel is now known as ykarel|lunch | 07:51 | |
| *** ykarel|meeting has quit IRC | 07:52 | |
| *** suuuper has joined #tripleo | 07:53 | |
| *** lucasagomes has joined #tripleo | 07:56 | |
| *** avivgta has quit IRC | 07:59 | |
| nawar | https://bugs.launchpad.net/tripleo/+bug/1840748 | 08:09 |
| openstack | Launchpad bug 1840748 in tripleo "undercloud deployment failed" [Undecided,New] | 08:09 |
| *** avivgta has joined #tripleo | 08:10 | |
| *** apetrich has joined #tripleo | 08:13 | |
| *** pkopec has joined #tripleo | 08:15 | |
| *** jaosorior has quit IRC | 08:16 | |
| *** panda has quit IRC | 08:18 | |
| *** panda has joined #tripleo | 08:20 | |
| *** ratailor_ has joined #tripleo | 08:24 | |
| openstackgerrit | Merged openstack/puppet-tripleo master: Make the rabbitmq-ready exec more stringent https://review.opendev.org/676431 | 08:25 |
| openstackgerrit | Cédric Jeanneret (Tengu) proposed openstack/tripleo-heat-templates master: Use tripleo-validations-package role instead of puppet https://review.opendev.org/677429 | 08:25 |
| *** bhagyashris has joined #tripleo | 08:25 | |
| openstackgerrit | Cédric Jeanneret (Tengu) proposed openstack/tripleo-heat-templates master: Use tripleo-validations-package role instead of puppet https://review.opendev.org/677429 | 08:26 |
| Tengu | pffff.... | 08:26 |
| *** ratailor has quit IRC | 08:27 | |
| *** derekh has joined #tripleo | 08:28 | |
| openstackgerrit | Keigo Noha proposed openstack/tripleo-heat-templates stable/queens: keystone: drop duplicate -DFOREGROUND https://review.opendev.org/677430 | 08:28 |
| *** ykarel|lunch is now known as ykarel | 08:29 | |
| dr_gogeta86 | morning all | 08:29 |
| *** yolanda has quit IRC | 08:30 | |
| *** tkajinam has quit IRC | 08:32 | |
| *** holser has joined #tripleo | 08:34 | |
| dr_gogeta86 | i didn't find any info for queens about multipath on containers | 08:38 |
| dr_gogeta86 | there is the client ... but how to configure it properly | 08:38 |
| *** yolanda has joined #tripleo | 08:43 | |
| openstackgerrit | Rabi Mishra proposed openstack/tripleo-heat-templates master: Move GroupVars to overcloud.j2.yaml https://review.opendev.org/677218 | 08:54 |
| *** shyamb has joined #tripleo | 08:55 | |
| *** cfontain has joined #tripleo | 08:59 | |
| openstackgerrit | Maciej Józefczyk proposed openstack/tripleo-heat-templates master: Explicitly disable DVR for OVN HA deployment https://review.opendev.org/663255 | 08:59 |
| *** jchhatbar has joined #tripleo | 09:06 | |
| *** jchhatbar has quit IRC | 09:06 | |
| *** jchhatbar has joined #tripleo | 09:07 | |
| *** janki has quit IRC | 09:07 | |
| *** jchhatbar has quit IRC | 09:07 | |
| *** janki has joined #tripleo | 09:08 | |
| *** rcernin has joined #tripleo | 09:16 | |
| openstackgerrit | Merged openstack/tripleo-ci master: Fix dumping variables when no featureset is defined https://review.opendev.org/677063 | 09:21 |
| shyamb | Hi | 09:22 |
| chkumar|rover | sshnaidm|afk: ^6 finally got merged | 09:22 |
| shyamb | Does RHOSP13 containers get updated after GA releases | 09:22 |
| sshnaidm|afk | chkumar|rover, unbelievable! | 09:23 |
| openstackgerrit | Sorin Sbarnea proposed openstack/tripleo-ci master: Configure ovb jobs to use ansible 2.8 https://review.opendev.org/677256 | 09:23 |
| *** apetrich has quit IRC | 09:25 | |
| shyamb | I see, RHOSP13 containers get updated | 09:26 |
| shyamb | My doubt is do we need this document for minor updates?: https://access.redhat.com/documentation/en-us/red_hat_openstack_platform/13/html-single/keeping_red_hat_openstack_platform_updated/index#assembly-upgrading_the_undercloud | 09:26 |
| *** apetrich has joined #tripleo | 09:28 | |
| *** waleedm has quit IRC | 09:30 | |
| *** shyamb has quit IRC | 09:45 | |
| *** nawar has left #tripleo | 09:46 | |
| *** jaosorior has joined #tripleo | 09:47 | |
| *** sshnaidm|afk is now known as sshnaidm | 09:49 | |
| *** ratailor__ has joined #tripleo | 09:53 | |
| openstackgerrit | Maciej Józefczyk proposed openstack/tripleo-heat-templates master: Explicitly disable DVR for OVN HA deployment https://review.opendev.org/663255 | 09:54 |
| *** holser has quit IRC | 09:55 | |
| *** mkisielewski has joined #tripleo | 09:55 | |
| *** ratailor_ has quit IRC | 09:56 | |
| *** holser has joined #tripleo | 09:56 | |
| *** shyamb has joined #tripleo | 09:59 | |
| *** gfidente has joined #tripleo | 10:05 | |
| *** bhagyashris has quit IRC | 10:06 | |
| *** pierreprinetti has quit IRC | 10:07 | |
| dr_gogeta86 | there is any module to configure multipathd | 10:08 |
| dr_gogeta86 | without modifying overcloud image | 10:08 |
| *** jpich has joined #tripleo | 10:12 | |
| openstackgerrit | Merged openstack/tripleo-validations stable/stein: Convert tox to native zuul https://review.opendev.org/677150 | 10:14 |
| *** egonzalez has quit IRC | 10:15 | |
| *** udesale has quit IRC | 10:16 | |
| *** udesale has joined #tripleo | 10:17 | |
| openstackgerrit | Merged openstack/tripleo-validations stable/stein: Adding molecule tests for no-op-firewall-nova-driver validation https://review.opendev.org/677171 | 10:24 |
| openstackgerrit | Merged openstack/tripleo-validations stable/stein: Adds molecule tests for the nova-status validation https://review.opendev.org/677172 | 10:24 |
| *** florianf has quit IRC | 10:25 | |
| *** morazi has quit IRC | 10:28 | |
| *** rcernin has quit IRC | 10:28 | |
| *** shyamb has quit IRC | 10:29 | |
| *** florianf has joined #tripleo | 10:30 | |
| *** jpich has quit IRC | 10:34 | |
| *** ykarel is now known as ykarel|afk | 10:34 | |
| *** ykarel|afk is now known as ykarel | 10:35 | |
| *** jpich has joined #tripleo | 10:36 | |
| *** gkadam has joined #tripleo | 10:38 | |
| mkisielewski | Hello everyone. On current-tripleo-rdo (Stein) I'm having some issues with running `openstack-cinder-volume`, after without-error deployment bundle under pcs control isn't started. After paunch debug (step 5) for `cinder_volume_init_bundle` and `cinder_volume_restart_bundle` I've found that in restart bundle `pcs resource restart` fails with `Error: Error performing operation: No such device or address` as | 10:53 |
| mkisielewski | openstack-cinder volume is not running anywhere. | 10:53 |
| *** waleedm has joined #tripleo | 10:58 | |
| *** gkadam is now known as gkadam-afk | 11:00 | |
| *** shyamb has joined #tripleo | 11:04 | |
| *** tesseract has joined #tripleo | 11:08 | |
| *** jpich has quit IRC | 11:08 | |
| *** paramite has joined #tripleo | 11:08 | |
| *** jpich has joined #tripleo | 11:08 | |
| *** ykarel is now known as ykarel|afk | 11:13 | |
| openstackgerrit | waleed mousa proposed openstack/os-net-config master: fix missing shared_block for PF interface https://review.opendev.org/670740 | 11:17 |
| *** udesale has quit IRC | 11:18 | |
| *** amoralej is now known as amoralej|lunch | 11:22 | |
| *** rh-jelabarre has joined #tripleo | 11:33 | |
| openstackgerrit | Natal Ngétal proposed openstack/tripleo-validations stable/stein: Add serial option in tripleo-ansible-inventory script https://review.opendev.org/660904 | 11:34 |
| openstackgerrit | Cédric Jeanneret (Tengu) proposed openstack/tripleo-ansible master: Adds tripleo-clients-install role https://review.opendev.org/677452 | 11:39 |
| *** ansmith_ has quit IRC | 11:40 | |
| *** gkadam-afk is now known as gkadam | 11:42 | |
| openstackgerrit | Chandan Kumar (raukadah) proposed openstack/tripleo-quickstart master: Use config_drive for network basicops tests https://review.opendev.org/676439 | 11:42 |
| *** morazi has joined #tripleo | 11:44 | |
| *** florianf has quit IRC | 11:46 | |
| openstackgerrit | waleed mousa proposed openstack/os-net-config master: fix missing shared_block for PF interface in switchdev bond connected to ovs https://review.opendev.org/670740 | 11:53 |
| openstackgerrit | waleed mousa proposed openstack/os-net-config master: fix missing shared_block for PF interface in switchdev bond connected to ovs https://review.opendev.org/670740 | 11:55 |
| *** shyam89 has joined #tripleo | 11:56 | |
| *** shyamb has quit IRC | 12:00 | |
| *** shyam89 has quit IRC | 12:01 | |
| *** cfontain has quit IRC | 12:02 | |
| *** rlandy has joined #tripleo | 12:02 | |
| *** raildo has joined #tripleo | 12:02 | |
| *** rlandy is now known as rlandy|ruck | 12:03 | |
| *** rpittau|afk is now known as rpittau | 12:05 | |
| *** ykarel|afk is now known as ykarel | 12:08 | |
| *** surpatil has quit IRC | 12:10 | |
| *** waleedm_ has joined #tripleo | 12:11 | |
| *** waleedm__ has joined #tripleo | 12:12 | |
| *** waleedm has quit IRC | 12:13 | |
| *** waleedm_ has quit IRC | 12:15 | |
| *** ratailor__ has quit IRC | 12:27 | |
| openstackgerrit | Chandan Kumar (raukadah) proposed openstack/tripleo-quickstart master: [DNM]Increase overcloud deploy timeout to 110 https://review.opendev.org/677458 | 12:31 |
| chkumar|rover | rlandy|ruck: ^^ | 12:31 |
| *** gkadam has quit IRC | 12:32 | |
| openstackgerrit | Cédric Jeanneret (Tengu) proposed openstack/tripleo-heat-templates master: Use tripleo-validations-package role instead of puppet https://review.opendev.org/677429 | 12:33 |
| *** bfournie has quit IRC | 12:35 | |
| mkisielewski | What's more interesting, I'm able to run cinder-volume container manually with the same config as defined in pcs. Where should I look to troubleshoot it as overcloud deployment finished successfully? | 12:36 |
| rlandy|ruck | chkumar|rover: k - good - let's watch that | 12:37 |
| *** cdearborn has joined #tripleo | 12:38 | |
| *** jaosorior has quit IRC | 12:43 | |
| *** ansmith_ has joined #tripleo | 12:44 | |
| Tengu | folks, quick question for Stein CI: https://object-storage-ca-ymq-1.vexxhost.net/v1/86bbbcfa8ad043109d2d7af530225c72/logs_28/675828/3/check/tripleo-buildimage-overcloud-full-centos-7/aef1b1e/job-output.txt - I see TASK [prepare-node : Dump hostvars on the host to pass it to inner ansible] | 12:49 |
| Tengu | I think that was reverted on master at some point... ? | 12:50 |
| Tengu | unless it's related to the job description, since it's talking about "The task includes an option with an undefined variable. The error was: 'featureset' is undefined" | 12:50 |
| *** florianf has joined #tripleo | 12:50 | |
| Tengu | chkumar|rover: -^^ does it ring any bells? | 12:51 |
| chkumar|rover | Tengu: the fix got merged few hours back | 12:51 |
| Tengu | chkumar|rover: in Stein? so if I recheck it should be good? | 12:51 |
| chkumar|rover | Tengu: https://review.opendev.org/#/c/677063/ | 12:52 |
| Tengu | ah, master | 12:52 |
| Tengu | we might want it in stein as well.. ? | 12:52 |
| chkumar|rover | Tengu: it is a tripleo-ci fix, so might work on all branches | 12:53 |
| Tengu | lemme recheck then :) | 12:54 |
| rlandy|ruck | rebase recheck? | 12:57 |
| *** ramishra has quit IRC | 12:57 | |
| *** jaosorior has joined #tripleo | 12:59 | |
| *** ramishra has joined #tripleo | 13:03 | |
| *** cfontain has joined #tripleo | 13:03 | |
| *** amoralej|lunch is now known as amoralej | 13:04 | |
| weshay | mwhahaha no rush.. checking if https://bugs.launchpad.net/tripleo/+bug/1826612 can be closed | 13:05 |
| openstack | Launchpad bug 1826612 in tripleo "nova placement code has been removed from puppet-nova which breaks puppet-tripleo testing" [Critical,In progress] - Assigned to Alex Schultz (alex-schultz) | 13:05 |
| *** ramishra has quit IRC | 13:05 | |
| openstackgerrit | Lukas Bezdicka proposed openstack/tripleo-upgrade stable/queens: Generate the network data and roles via jq https://review.opendev.org/670346 | 13:06 |
| *** owalsh has quit IRC | 13:09 | |
| *** waleedm__ has quit IRC | 13:12 | |
| *** waleedm has joined #tripleo | 13:12 | |
| *** florianf has quit IRC | 13:19 | |
| *** artom has quit IRC | 13:24 | |
| hjensas | fultonj: gfidente: ceph-ansible core's are'nt too happy about my proposed fix for https://bugs.launchpad.net/tripleo/+bug/1840030. If this was a puppet service I'd use hiera interpolation to get the CIDR of the node. Is there a way to do something like that in ansible? | 13:28 |
| openstack | Launchpad bug 1840030 in tripleo "OC deployment with multiple subnets under default networks fails with: msg: 'ipaddr: unknown filter type: 172.120.3.0/24,172.117.3.0/24,172.118.3.0/24,172.119.3.0/24'"" [High,In progress] - Assigned to Harald Jensås (harald-jensas) | 13:28 |
| *** artom has joined #tripleo | 13:28 | |
| *** ykarel is now known as ykarel|away | 13:33 | |
| mwhahaha | weshay: closed | 13:33 |
| weshay | thanks | 13:34 |
| *** ade_lee has quit IRC | 13:35 | |
| *** udesale has joined #tripleo | 13:35 | |
| *** ekultails has joined #tripleo | 13:39 | |
| *** ykarel|away has quit IRC | 13:40 | |
| *** owalsh has joined #tripleo | 13:41 | |
| *** Goneri has joined #tripleo | 13:41 | |
| abishop | dr_gogeta86: see https://github.com/openstack/tripleo-heat-templates/blob/master/environments/multipathd.yaml | 13:42 |
| *** udesale has quit IRC | 13:42 | |
| *** udesale has joined #tripleo | 13:43 | |
| *** aakarsh has quit IRC | 13:45 | |
| dr_gogeta86 | in queens is a litle bit different | 13:48 |
| dr_gogeta86 | i need to backport somethin | 13:48 |
| *** waleedm has quit IRC | 13:48 | |
| *** udesale has quit IRC | 13:50 | |
| dr_gogeta86 | abishop, ^ | 13:51 |
| abishop | dr_gogeta86: yes, queens is quite different (lots of things moved around) | 13:55 |
| abishop | dr_gogeta86: the current code on train has not been thoroughly tested, so not sure what to say about potential backports | 13:56 |
| *** ykarel|away has joined #tripleo | 13:59 | |
| *** ykarel|away is now known as ykarel | 13:59 | |
| abishop | dr_gogeta86: if you want to track the details, see https://review.opendev.org/#/q/topic:bug/1834042 | 14:00 |
| *** florianf has joined #tripleo | 14:00 | |
| fultonj | hjensas: i'll add looking at your patch to my list and get back to you | 14:00 |
| mwhahaha | #startmeeting tripleo | 14:00 |
| mwhahaha | #topic agenda | 14:00 |
| mwhahaha | * Review past action items | 14:00 |
| mwhahaha | * One off agenda items | 14:00 |
| mwhahaha | * Squad status | 14:00 |
| mwhahaha | * Bugs & Blueprints | 14:00 |
| mwhahaha | * Projects releases or stable backports | 14:00 |
| openstack | Meeting started Tue Aug 20 14:00:33 2019 UTC and is due to finish in 60 minutes. The chair is mwhahaha. Information about MeetBot at http://wiki.debian.org/MeetBot. | 14:00 |
| mwhahaha | * Specs | 14:00 |
| mwhahaha | * open discussion | 14:00 |
| mwhahaha | Anyone can use the #link, #action and #info commands, not just the moderatorǃ | 14:00 |
| mwhahaha | Hi everyone! who is around today? | 14:00 |
| openstack | Useful Commands: #action #agreed #help #info #idea #link #topic #startvote. | 14:00 |
| *** openstack changes topic to " (Meeting topic: tripleo)" | 14:00 | |
| openstack | The meeting name has been set to 'tripleo' | 14:00 |
| fultonj | o/ | 14:00 |
| *** openstack changes topic to "agenda (Meeting topic: tripleo)" | 14:00 | |
| ksambor | o/ | 14:00 |
| ykarel | o/ | 14:01 |
| ekultails | o/ | 14:01 |
| hjensas | fultonj: ok, thanks. | 14:01 |
| weshay | 0/ | 14:01 |
| rfolco | o/ | 14:02 |
| mschuppert | o/ | 14:02 |
| panda | o/ | 14:02 |
| rlandy|ruck | o/ | 14:02 |
| sshnaidm | \o | 14:02 |
| pkopec | o/ | 14:02 |
| Tengu | «o/ | 14:03 |
| mwhahaha | alright looks like we've got a group, let's go | 14:03 |
| mwhahaha | #topic review past action items | 14:03 |
| mwhahaha | None | 14:03 |
| *** openstack changes topic to "review past action items (Meeting topic: tripleo)" | 14:03 | |
| mwhahaha | #topic one off agenda items | 14:04 |
| mwhahaha | #link https://etherpad.openstack.org/p/tripleo-meeting-items | 14:04 |
| *** openstack changes topic to "one off agenda items (Meeting topic: tripleo)" | 14:04 | |
| *** janki has quit IRC | 14:04 | |
| mwhahaha | ykarel | 14:04 |
| mwhahaha | ocata and pike eol followup from last week | 14:04 |
| openstackgerrit | Merged openstack/puppet-tripleo master: Ensure Barbican required roles are created by Keystone https://review.opendev.org/675131 | 14:04 |
| mwhahaha | #link http://lists.openstack.org/pipermail/openstack-discuss/2019-August/008486.html | 14:04 |
| ykarel | yes, so as per Seans reply, Patch similar to https://review.opendev.org/#/c/583856/ in openstack/releases should be enough. | 14:04 |
| ykarel | so we can propose patch for ocata and pike eol and get feedback on reviews | 14:05 |
| mwhahaha | k | 14:05 |
| ykarel | there is still some confusion regarding automated parts, for that waiting for tonyb response | 14:05 |
| mwhahaha | please add me to any reviews and i'll +1 | 14:05 |
| ykarel | but good to propose patch to move forward | 14:05 |
| ykarel | okk than i will propose it starting with ocata | 14:06 |
| mwhahaha | sounds good | 14:06 |
| mwhahaha | shall we move on to the other topic? | 14:07 |
| chkumar|rover | \o/ | 14:08 |
| mwhahaha | alright moving on | 14:08 |
| mwhahaha | Removing Telemetry related jobs from TripleO CI | 14:08 |
| rh-jelabarre | o/ | 14:08 |
| mwhahaha | so which jobs? | 14:08 |
| Tengu | oh, can we take them back for validations ???? pleaaaaase?? :) | 14:09 |
| mwhahaha | you keep your dirty validations out of this | 14:09 |
| chkumar|rover | mwhahaha: fs05. 06 and 16 and 17 | 14:09 |
| mwhahaha | i have no idea which ones those are | 14:09 |
| chkumar|rover | and we run some telemetry tests in fs020 | 14:09 |
| mwhahaha | also i don't think telemetry is removed? | 14:09 |
| mwhahaha | i think we need some sort of announcement if telemetry bits are deprecated for soemthign else | 14:10 |
| ykarel | isn't telemetry services enabled by default? why only some featuresets | 14:10 |
| sshnaidm | fs16 and fs17 are scenarios 001 and 002 multinode container | 14:11 |
| chkumar|rover | it is going to be replaced with some service framework based on prometheus | 14:11 |
| ykarel | hmm they are default for overcloud https://github.com/openstack/tripleo-heat-templates/blob/master/overcloud-resource-registry-puppet.j2.yaml#L263-L267 | 14:11 |
| chkumar|rover | I donot know about the announcements | 14:11 |
| ykarel | need to check if it's explicitly disabled in ci | 14:11 |
| mwhahaha | i thought parts of it are still required for prometheus bits | 14:12 |
| mwhahaha | so i'm not sure about the call to remove services | 14:12 |
| openstackgerrit | Kevin Carter (cloudnull) proposed openstack/tripleo-heat-templates master: Convert firewall rules to use TripleO-Ansible https://review.opendev.org/677237 | 14:12 |
| mwhahaha | so whowever is driving this needs to raise the topic via the ML for discussion | 14:12 |
| chkumar|rover | mwhahaha: I asked this topic as we disabled telemetry tempest tests in ci very long time as no one is looking into it so asked | 14:13 |
| ykarel | yes good to have some background over it | 14:13 |
| chkumar|rover | *raised | 14:13 |
| *** ade_lee has joined #tripleo | 14:13 | |
| mwhahaha | my understand is that we still test it in a job somewhere | 14:13 |
| mwhahaha | and unless we're offically dropping support, we should have at least one job | 14:13 |
| *** aakarsh has joined #tripleo | 14:14 | |
| mwhahaha | if we lost the job in the standalone shuffle or something, that seems like it needs to be corrected | 14:14 |
| chkumar|rover | mwhahaha: ok, thanks, I will check with current telemetry dfg about the status. | 14:16 |
| mwhahaha | ok thanks | 14:16 |
| mwhahaha | moving on | 14:17 |
| mwhahaha | #topic Squad status | 14:17 |
| mwhahaha | ci | 14:17 |
| mwhahaha | #link https://etherpad.openstack.org/p/tripleo-ci-squad-meeting | 14:17 |
| mwhahaha | upgrade | 14:17 |
| mwhahaha | #link https://etherpad.openstack.org/p/tripleo-upgrade-squad-status | 14:17 |
| mwhahaha | edge | 14:17 |
| mwhahaha | #link https://etherpad.openstack.org/p/tripleo-edge-squad-status | 14:17 |
| *** openstack changes topic to "Squad status (Meeting topic: tripleo)" | 14:17 | |
| mwhahaha | integration | 14:17 |
| mwhahaha | #link https://etherpad.openstack.org/p/tripleo-integration-squad-status | 14:17 |
| mwhahaha | validations | 14:17 |
| mwhahaha | #link https://etherpad.openstack.org/p/tripleo-validations-squad-status | 14:17 |
| mwhahaha | networking | 14:17 |
| mwhahaha | #link https://etherpad.openstack.org/p/tripleo-networking-squad-status | 14:17 |
| mwhahaha | transformation | 14:17 |
| mwhahaha | #link https://etherpad.openstack.org/p/tripleo-ansible-agenda | 14:17 |
| mwhahaha | any status related highlights? | 14:17 |
| mwhahaha | alright movign on | 14:21 |
| mwhahaha | #topic bugs & blueprints | 14:21 |
| mwhahaha | #link https://launchpad.net/tripleo/+milestone/train-3 | 14:21 |
| mwhahaha | For Train we currently have 27 blueprints and 502 (-11) open Launchpad Bugs. 502 train-3. 172 (+0) open Storyboard bugs. | 14:21 |
| mwhahaha | #link https://storyboard.openstack.org/#!/project_group/76 | 14:21 |
| *** openstack changes topic to "bugs & blueprints (Meeting topic: tripleo)" | 14:21 | |
| mwhahaha | looks like we continue to make progress on bugs, thanks for closing some older ones out | 14:21 |
| mwhahaha | any bugs or blueprint highlights? | 14:21 |
| mwhahaha | sounds like no | 14:25 |
| mwhahaha | #topic projects releases or stable backports | 14:25 |
| *** openstack changes topic to "projects releases or stable backports (Meeting topic: tripleo)" | 14:25 | |
| mwhahaha | any one have nay backports that need eyes? | 14:25 |
| mwhahaha | milestone3 is the week of Sept 9 | 14:25 |
| mwhahaha | so make sure to lands all the things ASAP | 14:25 |
| mwhahaha | moving on | 14:29 |
| mwhahaha | #topic specs | 14:29 |
| mwhahaha | #link https://review.openstack.org/#/q/project:openstack/tripleo-specs+status:open | 14:29 |
| *** openstack changes topic to "specs (Meeting topic: tripleo)" | 14:29 | |
| mwhahaha | any comments on specs? | 14:29 |
| mwhahaha | Once we have a name for U i'll create the folder structure and lift my -2s so we can update specs | 14:30 |
| *** pkopec has quit IRC | 14:31 | |
| mwhahaha | #topic open discussion | 14:32 |
| *** openstack changes topic to "open discussion (Meeting topic: tripleo)" | 14:32 | |
| mwhahaha | anything else? | 14:32 |
| weshay | FYI.. looks like centos8 is in it's final stages https://wiki.centos.org/About/Building_8 | 14:33 |
| fultonj | yay | 14:33 |
| openstackgerrit | Merged openstack/tripleo-validations stable/stein: Adds molecule tests to image-serve and correct validation https://review.opendev.org/677197 | 14:33 |
| weshay | we have rhel8 standalone and ovb deploying in 3rd party.. working on a tempest (ping) issue but hopefully will make centos8 easier | 14:34 |
| weshay | that's all | 14:34 |
| mwhahaha | cool thanks | 14:34 |
| *** pkopec has joined #tripleo | 14:35 | |
| mwhahaha | last call for anything else | 14:35 |
| mwhahaha | alright thanks everyone | 14:37 |
| mwhahaha | #endmeeting | 14:37 |
| *** openstack changes topic to "CI Status: GREENish RDOCloud Status: MEHish | community irc meeting Tues@1400 UTC - tripleo-ci-community meeting Tues@1330 UTC | https://docs.openstack.org/tripleo-docs/latest/" | 14:37 | |
| openstack | Meeting ended Tue Aug 20 14:37:19 2019 UTC. Information about MeetBot at http://wiki.debian.org/MeetBot . (v 0.1.4) | 14:37 |
| openstack | Minutes: http://eavesdrop.openstack.org/meetings/tripleo/2019/tripleo.2019-08-20-14.00.html | 14:37 |
| openstack | Minutes (text): http://eavesdrop.openstack.org/meetings/tripleo/2019/tripleo.2019-08-20-14.00.txt | 14:37 |
| openstack | Log: http://eavesdrop.openstack.org/meetings/tripleo/2019/tripleo.2019-08-20-14.00.log.html | 14:37 |
| *** skramaja has quit IRC | 14:39 | |
| *** ykarel is now known as ykarel|away | 14:40 | |
| *** artom has quit IRC | 14:54 | |
| *** mkisielewski has quit IRC | 14:56 | |
| *** openstackstatus has quit IRC | 14:58 | |
| *** morazi has quit IRC | 14:58 | |
| *** openstack has joined #tripleo | 15:00 | |
| *** ChanServ sets mode: +o openstack | 15:00 | |
| *** openstackstatus has joined #tripleo | 15:00 | |
| *** ChanServ sets mode: +v openstackstatus | 15:00 | |
| *** ramishra has joined #tripleo | 15:01 | |
| *** yprokule has quit IRC | 15:01 | |
| openstackgerrit | Kevin Carter (cloudnull) proposed openstack/tripleo-common master: Log exceptions when checking status https://review.opendev.org/674919 | 15:02 |
| fultonj | can i get a +2 on https://review.opendev.org/#/c/673545/ which fixes scenario scenario010-multinode-oooq-container ? | 15:02 |
| fultonj | mwhahaha: ^ ? | 15:02 |
| mwhahaha | k | 15:02 |
| fultonj | thks | 15:03 |
| *** pkopec has quit IRC | 15:03 | |
| *** PagliaccisCloud has joined #tripleo | 15:03 | |
| *** pkopec has joined #tripleo | 15:03 | |
| dmsimard | cloudnull: o/ looking at the unauthorized issue predominantly showing up in ovh | 15:06 |
| cloudnull | ++ | 15:06 |
| *** chkumar|rover is now known as raukadah | 15:08 | |
| openstackgerrit | Merged openstack/tripleo-quickstart-extras master: Add overcloud deploy parameter to set CinderLVMLoopDeviceSize https://review.opendev.org/677227 | 15:10 |
| openstackgerrit | Merged openstack/tripleo-common-tempest-plugin master: Add Python 3 Train unit tests https://review.opendev.org/670656 | 15:10 |
| openstackgerrit | Merged openstack/tripleo-common-tempest-plugin master: Sync Sphinx requirement https://review.opendev.org/662391 | 15:10 |
| openstackgerrit | Merged openstack/tripleo-common-tempest-plugin master: Replace git.openstack.org URLs with opendev.org URLs https://review.opendev.org/654705 | 15:10 |
| openstackgerrit | Merged openstack/tripleo-puppet-elements master: Remove osops-tools-monitoring-oschecks for RHEL 8 https://review.opendev.org/676279 | 15:10 |
| mwhahaha | rlandy|ruck: fyi the buildah error log script doesn't work, https://openstack.fortnebula.com:13808/v1/AUTH_e8fd161dc34c421a979a9e6421f823e9/logs_97/676497/6/check/tripleo-build-containers-centos-7-buildah/fb1714a/logs/containers-build-errors.log.txt.gz | 15:11 |
| rlandy|ruck | k - will look at it | 15:11 |
| rlandy|ruck | thanks | 15:11 |
| mwhahaha | i also find all this excessive complexity terrible when trying to figure out what went wrong | 15:12 |
| mwhahaha | the old jobs was much better | 15:12 |
| dmsimard | cloudnull: I'm going to narrow my focus to "tripleo-ci-centos-7-standalone" which has the most failures right now | 15:14 |
| openstackgerrit | Bob Fournier proposed openstack/tripleo-puppet-elements stable/stein: Remove osops-tools-monitoring-oschecks for RHEL 8 https://review.opendev.org/677482 | 15:15 |
| cloudnull | dmsimard: from yesterday's conversation with stevebaker it would seem the docker api returns 401 when it really means 404 so there's some complexity there that we're likely going to need to handle. | 15:19 |
| dmsimard | cloudnull: why would that condition not be spread equally across providers, though ? | 15:20 |
| cloudnull | in https://review.opendev.org/674919 I've add A LOT of logging, and it would seem the connection starts to drop, we see "Resetting dropped connection" in the logs | 15:20 |
| dmsimard | there are some isolated occurrences in other providers but there is a definite bias towards ovh regions | 15:21 |
| cloudnull | likely due to some issue getting from OVH DC's to the docker CDN | 15:21 |
| dmsimard | first thing that came to mind was MTU misconfig | 15:21 |
| cloudnull | could be, but we only ever see the issue with one container image, rsyslog | 15:21 |
| dmsimard | only that one ? | 15:22 |
| cloudnull | maybe that container image is just larger than the rest ? | 15:22 |
| cloudnull | yup. just that one. seems its the last container image processed. | 15:22 |
| dmsimard | cloudnull: that's not rsyslog: http://logs.openstack.org/28/673328/5/gate/tripleo-ci-centos-7-scenario002-standalone/fceb2ff/logs/undercloud/var/log/tripleo-container-image-prepare.log.txt.gz?level=ERROR | 15:22 |
| dmsimard | this is from rdo cloud: https://logs.rdoproject.org/97/676497/6/openstack-check/tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset035/9c64197/logs/undercloud/var/log/tripleo-container-image-prepare.log.txt.gz#_2019-08-20_05_41_06_616 | 15:22 |
| cloudnull | like https://object-storage-ca-ymq-1.vexxhost.net/v1/86bbbcfa8ad043109d2d7af530225c72/logs_19/674919/21/check/tripleo-ci-centos-7-standalone/0176687/logs/undercloud/var/log/tripleo-container-image-prepare.log.txt.gz | 15:22 |
| *** artom has joined #tripleo | 15:23 | |
| cloudnull | check the last few reviews from - https://review.opendev.org/#/c/674919/ | 15:23 |
| cloudnull | In looking at the only tripleo-ci-centos-7-standalone the errors seem to always be rsyslog | 15:24 |
| dmsimard | ok let me check at the mirror/cache level to see if I can find something | 15:25 |
| openstackgerrit | Kevin Carter (cloudnull) proposed openstack/tripleo-common master: Log exceptions when checking status https://review.opendev.org/674919 | 15:26 |
| cloudnull | ^ that last review adds retries to the actual session via an adapter. maybe the issue is just a CDN cache miss. so my thinking is we we force the session to retry before raising an exception and letting tenacity rerun various methods. | 15:27 |
| cloudnull | though IDK, i'd be very grateful for more eyes on it. | 15:28 |
| dmsimard | cloudnull: http://paste.openstack.org/show/760317/ mirror access logs for https://object-storage-ca-ymq-1.vexxhost.net/v1/86bbbcfa8ad043109d2d7af530225c72/logs_19/674919/21/check/tripleo-ci-centos-7-standalone/0176687/logs/undercloud/var/log/tripleo-container-image-prepare.log.txt.gz | 15:35 |
| dmsimard | there's a minimal amount of proxy errors and the timestamps don't match, they do happen, though -- e.g, [2019-08-20 09:52:51.253429] [proxy:error] [pid 31236:tid 139663998179072] [client 23.253.166.88:49730] AH00898: Error reading from remote server returned by | 15:37 |
| dmsimard | /cloudflare/registry-v2/docker/registry/v2/blobs/sha256/be/be1dd21b990c16b4ae8aa5b8620fe408ee83cc54ffb5d80dea2ac9542af08273/data, | 15:37 |
| * dmsimard looks at ovh | 15:37 | |
| dmsimard | picked this one up: https://storage.gra1.cloud.ovh.net/v1/AUTH_dcaab5e32b234d56b626f72581e3644c/logs_97/676497/6/check/tripleo-ci-centos-7-standalone/b8c312d/logs/undercloud/var/log/tripleo-container-image-prepare.log.txt.gz bunch of 401's on centos-binary-nova-libvirt | 15:44 |
| cloudnull | dmsimard if you look for "_TripleOReAuth" you should be able to see if re-authenticate, and the number of times its happened. | 15:47 |
| dmsimard | cloudnull: there's a lot of 401's http://paste.openstack.org/show/760318/ | 15:48 |
| *** suuuper has quit IRC | 15:48 | |
| cloudnull | in review 674919 | 15:48 |
| cloudnull | yup | 15:48 |
| cloudnull | however, 401 could actually be 404. \ | 15:48 |
| cloudnull | to weed those out we added https://review.opendev.org/#/c/674919/23/tripleo_common/image/image_uploader.py@502 | 15:49 |
| dmsimard | that last paste got cropped | 15:49 |
| cloudnull | https://review.opendev.org/#/c/674919/23/tripleo_common/image/image_uploader.py@502 - that one ? | 15:50 |
| dmsimard | cloudnull, no mine | 15:50 |
| cloudnull | oh | 15:50 |
| dmsimard | non-cropped excerpt: http://paste.openstack.org/show/760319/ | 15:50 |
| cloudnull | it would seem that the docker API will return a specific error string when it encounters real token issues. | 15:51 |
| cloudnull | but you have to inspect the headers to get it | 15:51 |
| dmsimard | if I do a curl on the unproxied URL (i.e, https://registry-1.docker.io/v2/tripleomaster/centos-binary-nova-libvirt/blobs/sha256:c022ef85b64c565c31da47da01df95f1d784559244593bec0064de177bb00b66 ) I get a 401 | 15:53 |
| *** Lacquerhead has joined #tripleo | 15:53 | |
| dmsimard | {"errors":[{"code":"UNAUTHORIZED","message":"authentication required","detail":[{"Type":"repository","Class":"","Name":"tripleomaster/centos-binary-nova-libvirt","Action":"pull"}]}]} | 15:54 |
| cloudnull | yes you have to auth first | 15:56 |
| cloudnull | auth is just grabbing a token | 15:56 |
| cloudnull | just a sec I have a snippet | 15:56 |
| cloudnull | https://pasted.tech/pastes/c50c12c94e238ea29e51c153507904725906e9ea | 16:00 |
| *** udesale has joined #tripleo | 16:00 | |
| cloudnull | ^ That's mocking what the image_upload requests were doing | 16:00 |
| dmsimard | looking | 16:01 |
| *** ramishra has quit IRC | 16:02 | |
| *** cfontain has quit IRC | 16:02 | |
| dmsimard | in the meantime, I sanity checked that a docker pull on that particular image/hash works -> docker pull docker.io/tripleomaster/centos-binary-nova-libvirt:bb7bc41730bc5be97fe27a9445337152c45f08f6_a47d70e7 | 16:02 |
| cloudnull | I've experienced the same locally. | 16:02 |
| cloudnull | sadly ive never been able to reproduce the gate failure locally | 16:03 |
| cloudnull | even when using their proxy URLs | 16:03 |
| *** lucasagomes has quit IRC | 16:04 | |
| dmsimard | error="insufficient_scope" | 16:09 |
| dmsimard | there's www-authenticate Bearer realm="https://auth.docker.io/token",service="registry.docker.io",scope="repository:v2/tripleomaster/centos-binary-nova-compute:pull" | 16:10 |
| *** sanjayu__ has joined #tripleo | 16:10 | |
| dmsimard | and then www-authenticate Bearer realm="https://auth.docker.io/token",service="registry.docker.io",scope="repository:v2/tripleomaster/centos-binary-nova-compute:pull",error="insufficient_scope" | 16:11 |
| cloudnull | https://pasted.tech/pastes/8a76f5667b01a15c44e9ae9e88f95c8c58f833bd.raw - using the aforementioned snippet you should see something like so | 16:12 |
| cloudnull | scope="repository:v2/tripleomaster/centos-binary-nova-compute:pull" | 16:12 |
| dmsimard | yup, I got my output from the snipper | 16:12 |
| dmsimard | snippet* | 16:12 |
| dmsimard | need to try something | 16:13 |
| *** sanjayu_ has quit IRC | 16:13 | |
| dmsimard | cloudnull: I would tend to think that the scope shouldn't contain "v2/" | 16:14 |
| *** jpena is now known as jpena|off | 16:14 | |
| cloudnull | this was the URL I pulled from the logs | 16:16 |
| cloudnull | 'http://mirror.bhs1.ovh.openstack.org:8082/v2/tripleomaster/centos-binary-nova-compute/blobs/sha256:2efded40b28a63edb701aef3f646be560c1d938199334b173f496db2c | 16:16 |
| cloudnull | ...: a7285b1' | 16:16 |
| cloudnull | yuck | 16:16 |
| cloudnull | https://pasted.tech/pastes/69e4cbe920166cf4ba41f3e581f3341b8d79c3fa | 16:16 |
| *** rpittau is now known as rpittau|afk | 16:17 | |
| *** cylopez has quit IRC | 16:18 | |
| *** Vorrtex has joined #tripleo | 16:18 | |
| *** noama has quit IRC | 16:20 | |
| cloudnull | https://pasted.tech/pastes/3c725e2e72b263d4b4ac5c7093df565e8b9cd96f.raw | 16:21 |
| cloudnull | Attempting with a different image, same output. | 16:21 |
| openstackgerrit | Kevin Carter (cloudnull) proposed openstack/tripleo-common master: Log exceptions when checking status https://review.opendev.org/674919 | 16:23 |
| dmsimard | there's something going on with that v2/, trying to figure it out | 16:23 |
| *** morazi has quit IRC | 16:25 | |
| cloudnull | you only need that v2 when working with the openstack proxies, when using the docker hub URI directly you omit it - https://pasted.tech/pastes/bb10a5843318558ba5f7ec6953c78dd55e19bffd.raw | 16:26 |
| weshay | dmsimard++ cloudnull++ | 16:31 |
| weshay | thanks guys | 16:31 |
| cloudnull | did find a few github issues where folks say the registry URL should be changed from registry-1.docker.io ro registry.docker.io. | 16:32 |
| *** owalsh_ has joined #tripleo | 16:32 | |
| cloudnull | maybe that will help? though that will require some changes in infra to make that go | 16:32 |
| *** owalsh has quit IRC | 16:35 | |
| *** spsurya has quit IRC | 16:36 | |
| sshnaidm | cloudnull, I saw urls changes in logs between registry-1 and registry, didn't get the logic behind that.. | 16:37 |
| dmsimard | cloudnull: not getting the insufficient_scope error anymore | 16:37 |
| *** morazi has joined #tripleo | 16:38 | |
| dmsimard | https://gist.github.com/dmsimard/eea8812022cc9325bbf4c06e349f5705/revisions#diff-9d116d3acf19dddeedbf8a6af5998b59 from your original snippet | 16:38 |
| dmsimard | build_url was adding an extra v2 | 16:39 |
| dmsimard | from a previous test I was doing: | 16:39 |
| dmsimard | image_url: ParseResult(scheme='http', netloc='mirror.bhs1.ovh.openstack.org:8082', path='/v2/tripleomaster/centos-binary-nova-compute/blobs/sha256:2efded40b28a63edb701aef3f646be560c1d938199334b173f496db2ca7285b1', params='', query='', fragment='') | 16:39 |
| dmsimard | url: http://mirror.bhs1.ovh.openstack.org:8082/v2/v2/tripleomaster/centos-binary-nova-compute/blobs/sha256:2efded40b28a63edb701aef3f646be560c1d938199334b173f496db2ca7285b1 | 16:39 |
| dmsimard | for that run the scope was wrong: www-authenticate Bearer realm="https://auth.docker.io/token",service="registry.docker.io",scope="repository:v2/tripleomaster/centos-binary-nova-compute:pull" | 16:40 |
| dmsimard | what a proper scope seems to look like: www-authenticate Bearer realm="https://auth.docker.io/token",service="registry.docker.io",scope="repository:tripleomaster/centos-binary-nova-compute:pull" | 16:40 |
| *** owalsh_ is now known as owalsh | 16:55 | |
| openstackgerrit | Harry Rybacki proposed openstack/puppet-tripleo stable/stein: Ensure Barbican required roles are created by Keystone https://review.opendev.org/677515 | 16:59 |
| *** derekh has quit IRC | 16:59 | |
| *** udesale has quit IRC | 17:04 | |
| *** tesseract has quit IRC | 17:10 | |
| *** paramite has quit IRC | 17:12 | |
| openstackgerrit | Sergii Golovatiuk proposed openstack/tripleo-validations master: Set undercloud-connection to local by default https://review.opendev.org/653937 | 17:14 |
| openstackgerrit | Sergii Golovatiuk proposed openstack/tripleo-validations master: Add undercloud_key_file option in tripleo-ansible-inventory. https://review.opendev.org/676415 | 17:14 |
| *** ykarel|away has quit IRC | 17:18 | |
| *** kopecmartin is now known as kopecmartin|off | 17:19 | |
| openstackgerrit | David Moreau Simard proposed openstack/tripleo-common master: DNM: Print unhandled www_auth errors https://review.opendev.org/677521 | 17:19 |
| *** holser has quit IRC | 17:21 | |
| openstackgerrit | David Moreau Simard proposed openstack/tripleo-common master: DNM: Print unhandled www_auth errors https://review.opendev.org/677521 | 17:22 |
| *** ykarel|away has joined #tripleo | 17:22 | |
| *** raildo has quit IRC | 17:25 | |
| *** raildo has joined #tripleo | 17:32 | |
| weshay | http://logs.rdoproject.org/49/21649/39/check/periodic-tripleo-ci-rhel-8-ovb-3ctlr_1comp-featureset001-master/547cbde/ | 17:34 |
| weshay | first ovb, tripleo on rhel 8 full success :) | 17:34 |
| weshay | sshnaidm++ raukadah++ | 17:35 |
| sshnaidm | yay | 17:35 |
| openstackgerrit | David Moreau Simard proposed openstack/tripleo-common master: DNM: Print unhandled www_auth errors https://review.opendev.org/677521 | 17:36 |
| raukadah | great, we need to switch to os_tempest | 17:37 |
| *** ykarel|away has quit IRC | 17:42 | |
| openstackgerrit | Merged openstack/tripleo-heat-templates master: Switch scenario010-multinode-containers to bluestore https://review.opendev.org/673545 | 17:42 |
| *** amoralej is now known as amoralej|off | 17:43 | |
| *** sshnaidm is now known as sshnaidm|afk | 17:44 | |
| *** tesseract has joined #tripleo | 17:45 | |
| *** sanjayu__ has quit IRC | 17:46 | |
| *** florianf has quit IRC | 17:52 | |
| openstackgerrit | Merged openstack/tripleo-common stable/rocky: Adds redfish support to 'overcloud generate fencing'. https://review.opendev.org/677199 | 18:04 |
| *** morazi has quit IRC | 18:10 | |
| openstackgerrit | Luca Miccini proposed openstack/tripleo-common stable/rocky: Enable staging-ovirt (fence_rhevm) fencing agent. https://review.opendev.org/677531 | 18:13 |
| *** jpich has quit IRC | 18:18 | |
| openstackgerrit | Luca Miccini proposed openstack/tripleo-common stable/queens: Enable staging-ovirt (fence_rhevm) fencing agent. https://review.opendev.org/676420 | 18:21 |
| openstackgerrit | Merged openstack/tripleo-quickstart-extras master: Add rhel_containers variable https://review.opendev.org/676474 | 18:30 |
| openstackgerrit | Merged openstack/tripleo-common master: Changes to baremetal provision format https://review.opendev.org/669774 | 18:30 |
| openstackgerrit | Merged openstack/tripleo-common master: Break out hostname format logic into functions https://review.opendev.org/670432 | 18:30 |
| openstackgerrit | Merged openstack/puppet-tripleo master: Add certmonger-grafana-refresh script https://review.opendev.org/676395 | 18:32 |
| openstackgerrit | Merged openstack/diskimage-builder master: simple-init: add configurable RA timeout with network-manager https://review.opendev.org/677410 | 18:35 |
| openstackgerrit | Merged openstack/python-tripleoclient stable/stein: Allows to disable in-flight validations https://review.opendev.org/675828 | 18:46 |
| openstackgerrit | Harry Rybacki proposed openstack/tripleo-heat-templates stable/stein: Revert "Point InternalTLSVncCAFile to /etc/ipa/ca.crt" https://review.opendev.org/677548 | 18:55 |
| openstackgerrit | Harry Rybacki proposed openstack/tripleo-heat-templates master: Revert "Point InternalTLSVncCAFile to /etc/ipa/ca.crt" https://review.opendev.org/677549 | 18:56 |
| openstackgerrit | Harry Rybacki proposed openstack/tripleo-heat-templates stable/rocky: Revert "Point InternalTLSVncCAFile to /etc/ipa/ca.crt" https://review.opendev.org/677550 | 18:56 |
| *** Lacquerhead has quit IRC | 18:56 | |
| openstackgerrit | Harry Rybacki proposed openstack/tripleo-heat-templates stable/queens: Revert "Point InternalTLSVncCAFile to /etc/ipa/ca.crt" https://review.opendev.org/677551 | 18:56 |
| *** cjloader has left #tripleo | 19:10 | |
| openstackgerrit | Merged openstack/tripleo-common master: Add support for RHEL 8 and start using versionless element https://review.opendev.org/662693 | 19:27 |
| *** holser has joined #tripleo | 19:47 | |
| *** aakarsh has quit IRC | 19:53 | |
| sshnaidm|afk | mwhahaha, wrt https://review.opendev.org/#/c/676497/ - buildah failure is not related to patch, buildah job passed there in the same patchset: https://zuul.opendev.org/t/openstack/build/406612d9e3ff4de88f6f845641348665 | 19:55 |
| mwhahaha | i hadn't see it pass on that | 19:56 |
| mwhahaha | last two had been failures | 19:56 |
| openstackgerrit | Alex Schultz proposed openstack/tripleo-common master: Add "rhel_containers" variable to skip containers for RHEL https://review.opendev.org/676497 | 19:58 |
| *** ade_lee has quit IRC | 20:12 | |
| *** Goneri has quit IRC | 20:28 | |
| openstackgerrit | Ronelle Landy proposed openstack/tripleo-heat-templates master: Replace rpc-qdrouterd with rabbitmq-messaging-rpc https://review.opendev.org/677562 | 20:34 |
| openstackgerrit | Merged openstack/tripleo-common master: Add python3 file for hardened images https://review.opendev.org/676270 | 20:35 |
| openstackgerrit | Bob Fournier proposed openstack/tripleo-common stable/stein: Add python3 file for hardened images https://review.opendev.org/677564 | 20:36 |
| openstackgerrit | Bob Fournier proposed openstack/tripleo-common stable/stein: Add support for RHEL 8 and start using versionless element https://review.opendev.org/677565 | 20:37 |
| *** holser has quit IRC | 20:39 | |
| *** ansmith_ has quit IRC | 20:39 | |
| dmsimard | cloudnull: I wasn't able to reproduce the unauthorized issue but the rsyslog one is pretty consistent | 20:41 |
| cloudnull | with the resent changes I now see - https://pasted.tech/pastes/4f812b4099c8e2a5acaa9377b140f0c9d0f6f88b.raw | 20:42 |
| cloudnull | looks like there are some requests that are trying to send tls to a non-tls endpoint ? | 20:42 |
| cloudnull | this is the log file https://storage.bhs1.cloud.ovh.net/v1/AUTH_dcaab5e32b234d56b626f72581e3644c/logs_21/677521/3/check/tripleo-ci-centos-7-standalone/6406cc9/logs/undercloud/var/log/tripleo-container-image-prepare.log.txt.gz | 20:45 |
| dmsimard | cloudnull: is that new, though ? and why only rsyslog ? :( | 20:57 |
| *** holser has joined #tripleo | 20:57 | |
| cloudnull | its new log output | 20:58 |
| cloudnull | idk why rsyslog | 20:58 |
| openstackgerrit | Alex Schultz proposed openstack/python-tripleoclient master: Implement tripleo container image push command https://review.opendev.org/674800 | 21:02 |
| *** gfidente is now known as gfidente|afk | 21:07 | |
| openstackgerrit | Merged openstack/tripleo-docs master: Deprecate UI install instructions https://review.opendev.org/676720 | 21:08 |
| openstackgerrit | Merged openstack/tripleo-docs master: Migrate install to deploy-guide https://review.opendev.org/677029 | 21:08 |
| dmsimard | cloudnull: would holding a node help ? I'm otherwise out of ideas | 21:10 |
| cloudnull | it might ? | 21:10 |
| mwhahaha | maybe the rsyslog container is bad? | 21:11 |
| cloudnull | ^ | 21:11 |
| * mwhahaha invokes weshay | 21:11 | |
| weshay | I AM HERE LORD | 21:11 |
| cloudnull | curl of the cloudflare cdn returns - https://pasted.tech/pastes/b0544fd294d3d74806793732814fbcf3e8463586 | 21:11 |
| weshay | really... the rsyslog container is always the first one to be pulled afaict | 21:12 |
| cloudnull | though that could very well be due to age and authorization | 21:12 |
| * weshay looks | 21:12 | |
| weshay | mwhahaha while I have you.. if the deps for amqp are no longer available in rhel8 what then? we can drop support in train no? | 21:12 |
| weshay | no laughs at LORDY LORDY huh.. tough crowd | 21:13 |
| weshay | cloudnull so why do we think rsyslog is bad? on the one provider? | 21:13 |
| cloudnull | its failing on that one container a lot | 21:14 |
| *** ade_lee has joined #tripleo | 21:14 | |
| openstackgerrit | Merged openstack/tripleo-quickstart master: Use config_drive for network basicops tests https://review.opendev.org/676439 | 21:14 |
| openstackgerrit | Merged openstack/tripleo-heat-templates stable/queens: [FUP] Backport - krb-service-principal metadata per-Role https://review.opendev.org/664885 | 21:14 |
| weshay | ya.. I think that is coincidence not causality | 21:14 |
| weshay | maybe we can slow the promotions | 21:15 |
| weshay | https://hub.docker.com/r/tripleomaster/centos-binary-rsyslog/tags | 21:15 |
| weshay | so in theory if we were updating the containers.. say once a day.. do you guys think that could cause a problem? | 21:16 |
| weshay | hello, is this thing on? | 21:17 |
| dmsimard | weshay: the rsyslog image hasn't changed very recently | 21:18 |
| dmsimard | and is failing fairly consistently across different providers on this error: | 21:18 |
| dmsimard | msg="error pulling image \"docker://192.168.24.1:8787/tripleomaster/centos-binary-rsyslog:bb7bc41730bc5be97fe27a9445337152c45f08f6_a47d70e7\": Error reading blob sha256:eeed6d25dc3e17ce4389e478dbc4b5dc2f8148f9d22d7fb37991359feb98b31c: Digest did not match, expected sha256:eeed6d25dc3e17ce4389e478dbc4b5dc2f8148f9d22d7fb37991359feb98b31c, got | 21:18 |
| dmsimard | sha256:e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855" | 21:18 |
| dmsimard | exhibit A) limestone https://storage.bhs1.cloud.ovh.net/v1/AUTH_dcaab5e32b234d56b626f72581e3644c/logs_21/677521/3/check/tripleo-ci-centos-7-standalone/6406cc9/logs/undercloud/var/log/tripleo-container-image-prepare.log.txt.gz | 21:19 |
| weshay | dmsimard so when I look at the prep-containers log in /var/log... rsyslog is always the first one | 21:19 |
| * weshay looks at exhibit a | 21:19 | |
| cloudnull | ^ And that failure is using the `buildah` command | 21:19 |
| *** xek has quit IRC | 21:20 | |
| weshay | so limestone hits this how often | 21:20 |
| cloudnull | https://pasted.tech/pastes/3187f884c5df47d0aff333731d652bc797615d5b.raw - this is the first failure | 21:20 |
| dmsimard | I ran a skopeo inspect on that particular image/tag http://paste.openstack.org/raw/760468/ | 21:20 |
| *** ade_lee has quit IRC | 21:20 | |
| dmsimard | the expected hash is there, the unexpected one isn't (eh) | 21:20 |
| *** ade_lee has joined #tripleo | 21:21 | |
| openstackgerrit | Merged openstack/diskimage-builder master: update gentoo systemd profile to 17.1 from 17.0 https://review.opendev.org/677290 | 21:21 |
| weshay | dmsimard so if we upload new containers.. would the first container attempt at being pulled always miss? | 21:21 |
| weshay | because the sha wouldn't match | 21:21 |
| openstackgerrit | Kevin Carter (cloudnull) proposed openstack/tripleo-common master: Log exceptions when checking status https://review.opendev.org/674919 | 21:22 |
| dmsimard | weshay: it looks like some of the other images do work | 21:22 |
| *** aakarsh has joined #tripleo | 21:22 | |
| cloudnull | dmsimard added https://review.opendev.org/#/c/674919/25/tripleo_common/image/image_uploader.py@132 to see if we can get it to pass without the infra proxy bits. | 21:23 |
| cloudnull | when the scenario "tripleo-ci-centos-7-standalone" starts building, do you think we could hold that instance? | 21:24 |
| cloudnull | then we can login and try and further troubleshoot ? | 21:24 |
| cloudnull | maybe? | 21:24 |
| mwhahaha | weshay: no we need to fix the rhel8 deps (or wait until centos8) | 21:25 |
| mwhahaha | weshay: switch scenario003 to nv on rhel8 until deps are resolved would be solution | 21:25 |
| dmsimard | cloudnull: can't hold a specific node, I can only tell Zuul to hold the N next ones that fail for a particular job | 21:25 |
| weshay | ah k.. that's easy.. | 21:25 |
| weshay | rlandy|ruck ^ | 21:26 |
| dmsimard | let me add an autohold for it | 21:26 |
| weshay | mwhahaha so I should chase rel-del to build the package for el8 | 21:26 |
| mwhahaha | well i don't think they want to ship it in rhel | 21:26 |
| *** bnemec has quit IRC | 21:27 | |
| mwhahaha | but yea i would push them to solve that for centos8 at least | 21:27 |
| weshay | jjoyce fyi ^ | 21:27 |
| jjoyce | weshay: I would file a downstream bug against distribution with details for 16 and indicate that we need it for 15 as well. | 21:29 |
| mwhahaha | well i don't think we need it downstream was the problem | 21:29 |
| jjoyce | weshay: If you want to go the extra mile file 2 one for each release :-) | 21:29 |
| mwhahaha | we already hat this convo with amoralej | 21:29 |
| jjoyce | mwhahaha: Ah | 21:29 |
| mwhahaha | the problem is that we need it for upstream | 21:29 |
| jjoyce | Fun times | 21:29 |
| mwhahaha | and if we do rhel8 until centos8, we don't have it so our jobs break | 21:30 |
| openstackgerrit | Merged openstack/tripleo-puppet-elements stable/stein: Remove osops-tools-monitoring-oschecks for RHEL 8 https://review.opendev.org/677482 | 21:30 |
| weshay | is there any certainty that centos will build it? | 21:30 |
| * weshay looks | 21:30 | |
| mwhahaha | i don't know where we get it today | 21:30 |
| mwhahaha | we added amqp support but i don't know if we want to keep it or what | 21:31 |
| weshay | https://koji.mbox.centos.org/koji/search?match=glob&type=package&terms=python3-pyngus | 21:31 |
| dmsimard | cloudnull: the next two failures for tripleo-ci-centos-7-standalone should be held, will check every now and then to see if they are available | 21:31 |
| mwhahaha | it's to use qrouterd instead of rabbitmq | 21:31 |
| weshay | https://cbs.centos.org/koji/buildinfo?buildID=26196 | 21:31 |
| *** tesseract has quit IRC | 21:32 | |
| jjoyce | weshay: that may be python2 | 21:32 |
| weshay | jjoyce right.. I see it for python2, not python3 from centos | 21:32 |
| openstackgerrit | Merged openstack/python-tripleoclient master: Use reset to fix cmdline https://review.opendev.org/677296 | 21:33 |
| jjoyce | It's possible that version is not python3 compatible. | 21:33 |
| jjoyce | Though with a Jun build date I would expect someone to have looked at it. | 21:34 |
| weshay | is this something we have to carry? | 21:35 |
| jjoyce | Also odd that the naming did not change to python2-pyngus per the new naming conventions. | 21:35 |
| jjoyce | If you want to be sure you have the right version, it seems like something rdo will need to carry until you know that centos8 will have the correct version. | 21:36 |
| weshay | I suspect those guys are not going to build it | 21:36 |
| jjoyce | Very possible. | 21:36 |
| openstackgerrit | Alex Schultz proposed openstack/python-tripleoclient stable/stein: Use reset to fix cmdline https://review.opendev.org/677569 | 21:37 |
| *** raildo has quit IRC | 21:39 | |
| cloudnull | Thanks dmsimard | 21:40 |
| *** fultonj has quit IRC | 21:47 | |
| openstackgerrit | Merged openstack/python-tripleoclient master: Suppress output for ssh-keygen https://review.opendev.org/674631 | 21:49 |
| *** holser has quit IRC | 21:56 | |
| *** rcernin has joined #tripleo | 22:00 | |
| openstackgerrit | Ronelle Landy proposed openstack/tripleo-ci master: Move centos 7 buildah jobs to non-voting https://review.opendev.org/677575 | 22:11 |
| openstackgerrit | Steve Baker proposed openstack/tripleo-heat-templates master: Fixes for deploying nova-less undercloud https://review.opendev.org/677100 | 22:14 |
| *** zbr has quit IRC | 22:14 | |
| *** zbr has joined #tripleo | 22:15 | |
| openstackgerrit | Alex Schultz proposed openstack/python-tripleoclient stable/stein: Suppress output for ssh-keygen https://review.opendev.org/677577 | 22:19 |
| *** holser has joined #tripleo | 22:28 | |
| *** ansmith_ has joined #tripleo | 22:28 | |
| *** ekultails has quit IRC | 22:38 | |
| *** ksambor has quit IRC | 22:43 | |
| *** ksambor has joined #tripleo | 22:44 | |
| openstackgerrit | Alex Schultz proposed openstack/python-tripleoclient master: Implement tripleo container image push command https://review.opendev.org/674800 | 22:44 |
| *** PagliaccisCloud has quit IRC | 22:55 | |
| *** tkajinam has joined #tripleo | 22:55 | |
| tonyb | ykarel: Awesome! Thank you so much for the link to overcloud_containers.yaml.j2 That's exactly what I'm looking for | 22:57 |
| tonyb | ykarel: I'll reply to that thread tomorrow | 22:59 |
| tonyb | ykarel: sorry today ... I'm easily confused these days | 23:00 |
| openstackgerrit | Alex Schultz proposed openstack/python-tripleoclient master: Implement tripleo container image push command https://review.opendev.org/674800 | 23:09 |
| *** zbr has quit IRC | 23:22 | |
| *** zbr has joined #tripleo | 23:23 | |
| *** holser has quit IRC | 23:31 | |
| *** holser has joined #tripleo | 23:34 | |
| *** Vorrtex has quit IRC | 23:40 | |
| *** zaneb has quit IRC | 23:41 | |
| *** zaneb has joined #tripleo | 23:41 | |
| openstackgerrit | Merged openstack/puppet-tripleo stable/stein: Make the rabbitmq-ready exec more stringent https://review.opendev.org/677404 | 23:47 |
| openstackgerrit | Merged openstack/puppet-tripleo master: Add ability to install kernel related packages https://review.opendev.org/676503 | 23:47 |
| *** holser has quit IRC | 23:53 | |
| openstackgerrit | Merged openstack/tripleo-heat-templates stable/queens: Revert "Point InternalTLSVncCAFile to /etc/ipa/ca.crt" https://review.opendev.org/677551 | 23:53 |
Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!