Tuesday, 2020-02-04

« Monday, 2020-02-03 Index Wednesday, 2020-02-05 »
*** luksky has quit IRC00:22
*** spatel has joined #openstack-lbaas00:43
*** spatel has quit IRC00:48
johnsomSo, somehow we broke stable/stein, octavia-v2-dsvm-scenario-ubuntu-xenial, jobs. It is trying to use python3.6 but obviously xenial only had python2.7.....01:17
johnsomhttps://review.opendev.org/#/c/698597/01:17
openstackgerritMichael Johnson proposed openstack/octavia-tempest-plugin master: Fix octavia-v2-dsvm-scenario-ubuntu-xenial  https://review.opendev.org/70559001:22
johnsomrm_work Would you please take a look at https://review.opendev.org/705590 ?  stable/stein is broken.01:22
rm_worksure01:23
johnsomWe got a little over excited about the python3 transition.... lol01:23
johnsomIt goes boom: https://zuul.opendev.org/t/openstack/build/3516c4d67f3447f69792b97aecbb9589/log/job-output.txt#1320201:25
johnsomERROR: InterpreterNotFound: python3.601:25
*** mithilarun has quit IRC01:54
*** hongbin has joined #openstack-lbaas02:29
haleybjohnsom: i saw that failure in a stable/stein backport, i'd be happy if your change fixes it :)03:39
*** vishalmanchanda has joined #openstack-lbaas03:52
*** hongbin has quit IRC04:33
*** larsks has quit IRC05:05
*** squarebracket has quit IRC05:05
*** larsks has joined #openstack-lbaas05:12
*** squarebracket has joined #openstack-lbaas05:13
*** gcheresh has joined #openstack-lbaas05:53
openstackgerritMerged openstack/octavia-tempest-plugin master: Fix octavia-v2-dsvm-scenario-ubuntu-xenial  https://review.opendev.org/70559006:03
*** gcheresh has quit IRC06:08
*** gcheresh has joined #openstack-lbaas06:08
*** ltomasbo has joined #openstack-lbaas07:13
*** ccamposr has joined #openstack-lbaas07:29
*** mugsie has quit IRC07:37
*** mugsie has joined #openstack-lbaas07:41
*** yboaron_ has joined #openstack-lbaas07:48
*** AlexStaf has joined #openstack-lbaas07:49
*** yboaron_ has quit IRC07:55
*** yboaron_ has joined #openstack-lbaas07:55
*** yboaron_ has quit IRC08:03
*** yboaron_ has joined #openstack-lbaas08:04
*** yboaron_ has quit IRC08:04
*** vesper11 has quit IRC08:06
*** vesper11 has joined #openstack-lbaas08:07
*** maciejjozefczyk has joined #openstack-lbaas08:14
*** tkajinam has quit IRC08:17
*** luksky has joined #openstack-lbaas08:21
*** tesseract has joined #openstack-lbaas08:27
*** rpittau|afk is now known as rpittau08:33
*** gcheresh has quit IRC09:10
*** gcheresh has joined #openstack-lbaas09:10
brtknrrm_work: thanks09:39
dulekjohnsom: The Amp build issue I mentioned before happens consistently on all of our stable/train gates. It's not a hiccup.10:39
*** gcheresh has quit IRC10:50
*** gcheresh has joined #openstack-lbaas11:00
*** andy_ has quit IRC11:14
*** AlexStaf has quit IRC11:14
*** mugsie has quit IRC11:14
*** vishalmanchanda has quit IRC11:14
*** ltomasbo has quit IRC11:14
*** luketollefson has quit IRC11:14
*** gcheresh has quit IRC11:14
*** maciejjozefczyk has quit IRC11:14
*** squarebracket has quit IRC11:14
*** nmickus has quit IRC11:14
*** sapd1 has quit IRC11:14
*** strigazi has quit IRC11:14
*** canori01 has quit IRC11:14
*** logan- has quit IRC11:14
*** fyx has quit IRC11:14
*** dougwig has quit IRC11:14
*** sorrison has quit IRC11:14
*** irclogbot_2 has quit IRC11:15
*** openstackstatus has quit IRC11:16
*** irclogbot_3 has joined #openstack-lbaas11:16
*** luketollefson has joined #openstack-lbaas11:17
*** andy_ has joined #openstack-lbaas11:18
*** vesper11 has quit IRC11:18
*** ccamposr has quit IRC11:18
*** gthiemonge has quit IRC11:18
*** TMM has quit IRC11:18
*** dmellado has quit IRC11:18
*** TMM_ has joined #openstack-lbaas11:18
*** AlexStaf has joined #openstack-lbaas11:18
*** mugsie has joined #openstack-lbaas11:18
*** ltomasbo has joined #openstack-lbaas11:18
*** vishalmanchanda has joined #openstack-lbaas11:18
*** gcheresh has joined #openstack-lbaas11:18
*** maciejjozefczyk has joined #openstack-lbaas11:18
*** squarebracket has joined #openstack-lbaas11:18
*** nmickus has joined #openstack-lbaas11:18
*** sapd1 has joined #openstack-lbaas11:18
*** strigazi has joined #openstack-lbaas11:18
*** canori01 has joined #openstack-lbaas11:18
*** logan- has joined #openstack-lbaas11:18
*** fyx has joined #openstack-lbaas11:18
*** dougwig has joined #openstack-lbaas11:18
*** sorrison has joined #openstack-lbaas11:18
*** rm_work has quit IRC11:21
*** dmellado has joined #openstack-lbaas11:21
*** ccamposr has joined #openstack-lbaas11:21
*** mnaser has quit IRC11:21
*** rm_work has joined #openstack-lbaas11:22
*** gthiemonge has joined #openstack-lbaas11:22
*** vesper11 has joined #openstack-lbaas11:22
*** mnaser has joined #openstack-lbaas11:22
*** tesseract has quit IRC11:22
*** tesseract has joined #openstack-lbaas11:24
*** tesseract has quit IRC11:56
brtknrdoes neutron lbaas work on Train?12:09
cgoncalvesbrtknr, neutron-lbaas was not released in Train. last release was Stein12:10
brtknrcgoncalves: ah thanks12:15
*** tesseract has joined #openstack-lbaas12:26
*** vishalmanchanda has quit IRC12:29
*** tesseract has quit IRC12:29
*** tesseract has joined #openstack-lbaas12:31
*** tesseract has quit IRC12:37
*** tesseract has joined #openstack-lbaas12:38
*** tesseract has quit IRC12:40
*** tesseract has joined #openstack-lbaas12:40
*** tesseract has quit IRC12:50
*** tesseract has joined #openstack-lbaas13:03
*** tesseract has quit IRC13:07
*** tesseract has joined #openstack-lbaas13:15
*** luksky has quit IRC13:19
*** gcheresh has quit IRC13:22
*** tkajinam has joined #openstack-lbaas13:34
*** maciejjozefczyk has quit IRC13:36
*** gcheresh has joined #openstack-lbaas13:39
*** pcaruana has quit IRC13:43
*** maciejjozefczyk has joined #openstack-lbaas13:46
*** sapd1_x has joined #openstack-lbaas14:17
*** spatel has joined #openstack-lbaas14:19
*** spatel has quit IRC14:24
*** TrevorV has joined #openstack-lbaas14:32
*** tkajinam has quit IRC14:32
*** luksky has joined #openstack-lbaas14:44
*** vishalmanchanda has joined #openstack-lbaas14:49
*** gcheresh has quit IRC14:53
*** pcaruana has joined #openstack-lbaas14:56
*** sapd1_x has quit IRC15:10
*** gcheresh has joined #openstack-lbaas15:32
openstackgerritCarlos Goncalves proposed openstack/octavia stable/stein: DNM: testing fix for amphora image builds  https://review.opendev.org/70571315:32
*** Trevor_V has joined #openstack-lbaas15:43
johnsomdulek I just built a stable/train image locally, so the diskimage-builder project didn't break something. Can you provide a link the patch that is experiencing this?15:44
*** TrevorV has quit IRC15:44
*** Trevor_V is now known as TrevorV15:44
johnsom2020-02-04 15:43:42.840 | Build completed successfully15:44
johnsomSuccessfully built the amphora using the stable/train amphora-agent.15:44
dulekjohnsom: Any of those: https://review.opendev.org/#/q/project:openstack/kuryr-kubernetes+status:open+branch:stable/train.15:45
dulekjohnsom: I'll go over our Zuul settings…15:45
dulekjohnsom: Do you see anything weird or missing here: https://github.com/openstack/kuryr-kubernetes/blob/28a871a036492e8fd1487744243714bc137df595/.zuul.d/octavia.yaml#L15-L47 ?15:47
johnsomYeah, these are all odd:15:48
johnsomhttps://www.irccloud.com/pastebin/VDlP2b0O/15:48
johnsomImage size of 3 is only needed for centos/rhel images, but you aren't setting the centos variable.15:49
johnsomNo idea why you want to wait 9999 on nova to start the qemu process, but....15:49
dulekjohnsom: The first one is a leftover from us using prebuilt CentOS image.15:50
johnsomThat usually is pretty fast, seconds on a healthy cloud. It's the nova boot time variable that can take too long and poor clouds.15:50
dulekjohnsom: The second one is there for a looong time, probably fighting gate slowness with nested virt.15:50
johnsomdulek Yeah, that isn't even the nested virt timeout variable15:50
johnsomIt's just nova scheduling and qemu process start15:51
dulekjohnsom: Anyway that shouldn't matter too much as we've merged patches on Jan 15 with this.15:51
dulekAnd as I only recently noticed this I'd bet it's either a59e7235560ad045b24c58ed722ab62a451174d3 or db26a3c96b98d13d7da7cc5575c81308e0b7a09b on openstack/octavia?15:52
dulekBoth touch DIB?15:52
dulekjohnsom: If that doesn't ring a bell I can try reverting them and testing?15:54
johnsomYeah, we merged stable/train patches Friday, so a bit puzzled too. Let me dig into your job a bit more. I see you have an older DIB for some reason.15:54
johnsomdulek This isn't an Octavia bug for sure. The error is diskimage-builder before it hits our code.15:54
dulekjohnsom: Ah, interesting. So maybe it's db26a3c96b98d13d7da7cc5575c81308e0b7a09b as it seems to be reverted because a new version got released?15:55
dulekAnd we don't have it?15:55
*** ltomasbo has quit IRC15:57
johnsomYeah, your job has 2.27.0 from Sep 8, 2019. Not sure why as DIB is branchless15:57
dulekjohnsom: You have DIB on required-projects, so probably you're using git version. We don't and we use pip releases.15:58
johnsomdulek I didn't use the git release for my local test. See pypi: https://pypi.org/project/diskimage-builder/#history15:58
dulekjohnsom: Ah, but latest is 2.33.0…15:59
johnsomIt should have used 2.33.015:59
dulekjohnsom: https://github.com/openstack/requirements/blob/stable/train/upper-constraints.txt#L61216:00
dulekjohnsom: It's upper-constraints that block it.16:00
dulekAnd the idea is to not bump it lightly in stable branches.16:01
johnsomYeah, that is a problem as DIB has had breaking issues16:01
johnsomMaybe we should inquire in the DIB channel16:02
dulekSo there are 3 fixes - reverting db26a3c96b98d13d7da7cc5575c81308e0b7a09b, bumping upper-constraint or making kuryr-kubernetes use git version of DIB.16:02
dulekI'd say either of the 2 first is a correct one. Probably best to bump the constraint.16:02
dulekjohnsom: I just joined #openstack-dib.16:03
*** ltomasbo has joined #openstack-lbaas16:19
*** ltomasbo has quit IRC16:24
*** gcheresh has quit IRC16:30
*** maciejjozefczyk has quit IRC16:32
*** jamesdenton has joined #openstack-lbaas16:39
*** mithilarun has joined #openstack-lbaas16:42
*** mithilarun has quit IRC16:55
*** Trevor_V has joined #openstack-lbaas16:56
*** TrevorV has quit IRC16:59
*** tesseract has quit IRC17:04
*** luksky has quit IRC17:07
*** ccamposr has quit IRC17:22
*** rpittau is now known as rpittau|afk17:27
*** armax has quit IRC17:30
*** mithilarun has joined #openstack-lbaas17:31
*** mithilarun has quit IRC17:35
*** luksky has joined #openstack-lbaas17:50
*** TrevorV has joined #openstack-lbaas18:13
*** Trevor_V has quit IRC18:16
*** armax has joined #openstack-lbaas18:17
*** mithilarun has joined #openstack-lbaas18:26
*** mithilar_ has joined #openstack-lbaas18:27
*** mithilarun has quit IRC18:31
cgoncalvescan requirements be bumped on stable branches?18:36
cgoncalvesdoes kuryr assert for the stable policy tag?18:36
cgoncalvesnever mind. you said upper-constraints (requirements project). that would be a "no" I guess18:37
*** mithilar_ has quit IRC18:46
*** mithilarun has joined #openstack-lbaas18:46
*** amuller has joined #openstack-lbaas18:56
*** andy_ has quit IRC18:59
*** luketollefson has quit IRC19:00
*** andy_ has joined #openstack-lbaas19:01
*** luketollefson has joined #openstack-lbaas19:02
johnsomYeah, so I had a chat in the requirements channel. Since DIB is branch-less, we need to bump it in all of the stable UC files.19:09
*** mithilarun has quit IRC19:11
*** mithilarun has joined #openstack-lbaas19:12
*** mithilar_ has joined #openstack-lbaas19:17
*** mithilarun has quit IRC19:21
johnsomhaleyb For some reason your comment about the stable/train branch gives me the feeling you know more about this issue... lol. FYI, there is a conversation about it in the qa channel and they are working on a fix for the tempest tox env.19:37
*** vishalmanchanda has quit IRC19:53
haleybjohnsom: there was an issue with tempest on rocky, and part of the fix was to make things run in a 3.6 env, this might have been collateral damage.  i haven't been able to keep up with it all20:05
*** born2bake has joined #openstack-lbaas20:09
*** jamesdenton has quit IRC20:48
*** TrevorV has quit IRC20:57
johnsomYeah, me either21:34
*** armax has quit IRC21:40
*** luksky has quit IRC21:41
*** amuller has quit IRC21:46
*** spatel has joined #openstack-lbaas22:02
*** armax has joined #openstack-lbaas22:03
*** spatel has quit IRC22:06
johnsomdulek FYI, talking with the requirements team, the train issue is a requirements UC issue with DIB. I have proposed UC updates based on the feedback.22:23
*** mithilar_ has quit IRC22:32
haleybjohnsom: question for you. i was trying to rebase by "remove six" patch today and don't know if something snuck-in in the recent jobboard patches22:53
haleybhttps://opendev.org/openstack/octavia/src/branch/master/octavia/controller/worker/v1/tasks/database_tasks.py#L934 and https://opendev.org/openstack/octavia/src/branch/master/octavia/controller/worker/v2/tasks/database_tasks.py#L1004 are slightly different22:53
haleybthe v2 one has an encode("utf-8") but not the v1, and looking through the tree it's not consistent22:54
haleybthe encode change was introduced in dad38f61f823f5121f32b24ebcbd8703128dd32e22:55
johnsomI haven't been following the jobboard patches super closely, I think Adam and Carlos have been reviewing those.22:57
haleybjohnsom: ack, thanks22:58
johnsomI'm looking to see if I can see why that changed...22:58
haleybrm_work, cgoncalves ^^^ question above, i don't know if something snuck in that patch?22:58
johnsomhaleyb I guess this is a clue: https://review.opendev.org/#/c/668898/47/octavia/controller/worker/v2/tasks/cert_task.py22:59
johnsomI'm still not really following the *why* part. The DB should be setup to store utf-823:01
haleybjohnsom: oh, that would be it, i'll have to work backwards too see how removing six is breaking the unit tests for me. but it doesn't explain why the v1 code doesn't do it23:01
johnsomOh, it must be storing that in the new jobboard db... hmmm23:01
johnsomhaleyb Yeah, I think she is talking about the jobboard DB as in the legacy code, we don't store that data in mysql23:03
haleyblegacy == v1 ?23:03
johnsomhaleyb Yeah, sorry, making up my own terms23:04
haleybas long as what's there is right, i can fix my rebase nightmare to deal with it :)23:05
johnsomfernet usually takes bytes23:07
*** tkajinam has joined #openstack-lbaas23:09
haleybthe failure i am seeing is AttributeError: 'bytes' object has no attribute 'encode' :-/23:10
haleybclear as mud23:10
haleybunless it's something i did in the six removal changing something to the wrong type23:11
johnsomYeah, so whatever "server_pem" was passed in is already in bytes23:11
johnsomShe probably added that encode as it was 'str' type23:13
johnsomfor python323:13
haleybjohnsom: i think you've got something there, the unit tests were using get_six_compatible_value() which I removed, so this it probably my fault23:15
haleybjohnsom: thanks for the help... when i initially rebased i removed two decode() calls by accident :-o23:31
johnsom+123:31
*** mithilarun has joined #openstack-lbaas23:31
openstackgerritBrian Haley proposed openstack/octavia master: Remove all usage of six library  https://review.opendev.org/70129023:31
*** mithilarun has quit IRC23:31
*** mithilarun has joined #openstack-lbaas23:32
*** born2bake has quit IRC23:50
rm_workwould be good to get these in, as multi-AZ is very broken without them: https://review.opendev.org/#/q/topic:az-tweaks23:57
rm_workyou can hopefully understand why, just by looking at what they touch23:57
« Monday, 2020-02-03 Index Wednesday, 2020-02-05 »

Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!