Thursday, 2020-06-11

*** hamalq_ has quit IRC00:05
*** hongbin has quit IRC00:16
*** jmlowe has joined #openstack-nova00:24
*** ociuhandu has joined #openstack-nova00:24
*** ociuhandu has quit IRC00:29
*** hemna has quit IRC00:30
*** hemna has joined #openstack-nova00:30
*** jmlowe has quit IRC00:33
*** hongbin has joined #openstack-nova00:34
*** hemna has quit IRC00:38
*** hemna has joined #openstack-nova00:39
*** sapd1 has joined #openstack-nova00:51
*** sapd1 has quit IRC00:59
*** sapd1 has joined #openstack-nova01:04
*** tbachman has quit IRC01:14
*** tbachman has joined #openstack-nova01:15
*** lbragstad has joined #openstack-nova01:18
*** Liang__ has joined #openstack-nova01:18
*** sapd1 has quit IRC01:23
*** hongbin has quit IRC01:42
*** hemna has quit IRC01:52
*** hemna has joined #openstack-nova01:53
*** hemna has quit IRC01:53
*** hemna has joined #openstack-nova01:54
*** Liang__ has quit IRC02:06
*** Liang__ has joined #openstack-nova02:10
*** Liang__ has quit IRC02:15
*** Liang__ has joined #openstack-nova02:21
*** xinranwang has joined #openstack-nova02:28
*** jmlowe has joined #openstack-nova02:41
*** hongbin has joined #openstack-nova02:43
*** Liang__ has quit IRC02:44
openstackgerritsunhao proposed openstack/nova master: Add checks for volume status when rebuilding  https://review.opendev.org/73473402:46
*** spatel has joined #openstack-nova03:07
*** spatel has quit IRC03:08
*** rcernin has quit IRC03:08
*** tetsuro has quit IRC03:16
*** xiaolin has joined #openstack-nova03:31
*** lbragstad has quit IRC03:36
*** rcernin has joined #openstack-nova03:54
*** tetsuro has joined #openstack-nova03:54
*** rcernin has quit IRC03:55
*** rcernin has joined #openstack-nova03:55
*** yaawang has quit IRC04:11
*** yaawang has joined #openstack-nova04:11
*** evrardjp has quit IRC04:33
*** hongbin has quit IRC04:33
*** evrardjp has joined #openstack-nova04:33
openstackgerritsunhao proposed openstack/nova master: Add checks for volume status when rebuilding  https://review.opendev.org/73473404:44
*** links has joined #openstack-nova04:50
*** boxiang has quit IRC04:55
*** boxiang has joined #openstack-nova04:55
*** Liang__ has joined #openstack-nova04:56
*** gyee has quit IRC05:11
*** links has quit IRC05:12
*** links has joined #openstack-nova05:21
*** jaosorior has quit IRC05:21
*** jaosorior has joined #openstack-nova05:23
*** vishalmanchanda has joined #openstack-nova05:35
*** arne_wiebalck has quit IRC05:38
*** arne_wiebalck has joined #openstack-nova05:38
*** Liang__ has quit IRC05:38
*** Liang__ has joined #openstack-nova05:39
*** Liang__ has quit IRC05:43
*** Liang__ has joined #openstack-nova05:44
*** ratailor has joined #openstack-nova05:44
*** tetsuro_ has joined #openstack-nova05:45
*** jsuchome has joined #openstack-nova05:46
*** tetsuro__ has joined #openstack-nova05:47
*** tetsuro has quit IRC05:47
*** tetsuro_ has quit IRC05:50
*** yaawang has quit IRC05:55
*** yaawang has joined #openstack-nova05:56
*** udesale has joined #openstack-nova06:02
*** dklyle has quit IRC06:04
*** udesale has quit IRC06:17
*** udesale has joined #openstack-nova06:18
fricklersean-k-mooney: implemented a first draft of your idea regarding pip3 in https://review.opendev.org/735058 , tested only on focal so far, waiting for test results on other distros. ideally all of them would no longer need this whole manual installation thing anymore06:22
*** dustinc has quit IRC06:35
*** belmoreira has joined #openstack-nova06:38
*** jmlowe has quit IRC06:45
*** xinranwang has quit IRC06:54
*** yaawang has quit IRC06:55
*** yaawang has joined #openstack-nova06:56
*** tesseract has joined #openstack-nova07:05
*** yaawang has quit IRC07:08
*** yaawang has joined #openstack-nova07:09
*** rpittau|afk is now known as rpittau07:18
openstackgerritHarshavardhan Metla proposed openstack/nova master: Raising a failure in case of 403  https://review.opendev.org/73506807:18
*** ttsiouts has joined #openstack-nova07:19
*** ralonsoh has joined #openstack-nova07:21
*** jawad_axd has joined #openstack-nova07:21
*** ttsiouts has quit IRC07:22
*** ttsiouts has joined #openstack-nova07:24
openstackgerritMarek Lyčka proposed openstack/nova-specs master: Enable REST quiescing  https://review.opendev.org/70281007:25
*** boxiang_ has joined #openstack-nova07:30
*** avolkov has joined #openstack-nova07:31
*** boxiang has quit IRC07:34
*** tetsuro__ has quit IRC07:37
*** tosky has joined #openstack-nova07:38
*** tetsuro has joined #openstack-nova07:40
*** ociuhandu has joined #openstack-nova07:41
*** factor has quit IRC07:46
*** factor has joined #openstack-nova07:47
*** ociuhandu has quit IRC07:50
*** ociuhandu has joined #openstack-nova07:52
*** rcernin has quit IRC08:00
*** tesseract has quit IRC08:02
*** tesseract has joined #openstack-nova08:03
*** yaawang has quit IRC08:11
*** yaawang has joined #openstack-nova08:11
*** tesseract has quit IRC08:11
*** martinkennelly has joined #openstack-nova08:11
*** slaweq has quit IRC08:15
*** jamesdenton has quit IRC08:16
openstackgerritMarek Lyčka proposed openstack/nova-specs master: Enable REST quiescing  https://review.opendev.org/70281008:18
*** slaweq has joined #openstack-nova08:20
*** boxiang has joined #openstack-nova08:31
*** ociuhandu has quit IRC08:31
*** boxiang_ has quit IRC08:31
*** ociuhandu has joined #openstack-nova08:34
*** ociuhandu has quit IRC08:36
*** ociuhandu has joined #openstack-nova08:42
*** ociuhandu has quit IRC08:46
*** ociuhandu has joined #openstack-nova08:47
*** tesseract has joined #openstack-nova08:49
*** rcernin has joined #openstack-nova08:50
*** ociuhandu has quit IRC08:53
*** tesseract-RH has joined #openstack-nova08:56
openstackgerritAlexandre Arents proposed openstack/nova stable/stein: Reserve DISK_GB resource for the image cache  https://review.opendev.org/73252908:56
*** tesseract-RH has quit IRC08:56
*** tesseract has quit IRC08:56
*** tesseract has joined #openstack-nova08:56
*** tetsuro has quit IRC08:57
*** martinkennelly has quit IRC09:00
*** tetsuro has joined #openstack-nova09:00
*** sapd1_x has quit IRC09:02
*** sapd1 has joined #openstack-nova09:06
bauzaslate morning hello world09:08
bauzaswas on teaching mode for my kid...09:09
*** rpittau is now known as rpittau|bbl09:15
*** factor has quit IRC09:16
*** factor has joined #openstack-nova09:16
*** rcernin has quit IRC09:17
gibibauzas: o/09:17
bauzasspecs writing and reviewing day FTW (hopefully)09:18
bauzasgibi: if you have some of them you want me to look...09:18
bauzasgibi: btw. tomorrow will be a Red Hat worlwide PTO09:18
bauzasgibi: so you won't see all of us I think09:18
gibibauzas: I looket throught the open specs, the list is short so I suggest to look there too09:19
bauzaskk09:19
gibibauzas: thanks for the heads up. Have a nice PTO tomorrow09:19
*** slaweq has quit IRC09:19
bauzasit's actually a company holiday09:19
gibiinteresting concept09:19
gibido you have company events on this day or it is simply a day off09:20
gibi?09:20
*** ociuhandu has joined #openstack-nova09:20
*** tetsuro has quit IRC09:28
*** ociuhandu has quit IRC09:28
bauzasgibi: my company calls it 'Recharge Day'09:29
bauzasgibi: US folks had company holidays called 'shutdown' between Xmas and New Year09:29
bauzasnow it's a bit changing for sales reasons, so they created 'Recharge days'09:30
bauzasand for the first time, they offered recharge days09:30
bauzasfor this quarter, the first holiday will be tomorrow but we'll have a Q3 and Q4 holiday too09:30
*** Liang__ has quit IRC09:31
*** ociuhandu has joined #openstack-nova09:32
gibisounds good. Local Ericsson has a yearly recharge week but thats a normal work week with some exta perks like healthy food and free medical checkups09:36
*** ratailor has quit IRC09:38
*** ratailor has joined #openstack-nova09:38
bauzasgibi: are you still working remotely or back at the office ?09:43
gibibauzas: still at home full time. I think from next week we can slowly return to the office but it is not mandatory to go back at this time09:46
bauzasack09:56
*** slaweq has joined #openstack-nova09:58
*** slaweq has quit IRC10:20
*** ociuhandu has quit IRC10:25
*** efried has quit IRC10:32
openstackgerritHarshavardhan Metla proposed openstack/nova master: Raising a failure in case of 403  https://review.opendev.org/73506810:35
*** jraju__ has joined #openstack-nova10:35
*** links has quit IRC10:36
*** ttsiouts has quit IRC10:39
*** martinkennelly has joined #openstack-nova10:42
*** ociuhandu has joined #openstack-nova10:42
*** derekh has joined #openstack-nova10:43
gibistephenfin: hi! as I see the first part of the impl of bp use-pcpu-and-vcpu-in-one-instance is from you and it seems ready for a Runway slot. Will the next two weeks good for you to iterate on those patches?10:46
*** sapd1 has quit IRC10:46
*** ociuhandu has quit IRC10:52
*** ociuhandu has joined #openstack-nova10:53
*** purplerbot has joined #openstack-nova10:58
*** ociuhandu has quit IRC10:58
*** ttsiouts has joined #openstack-nova11:03
stephenfingibi: yup, sure thing11:05
*** martinkennelly has quit IRC11:05
*** martinkennelly has joined #openstack-nova11:05
gibistephenfin: cool. I will bring it up on the weekly meeting so if there are cores who willing to spend time on reviewing it11:05
stephenfin \o/ sounds good to me11:06
*** yaawang has quit IRC11:08
*** yaawang has joined #openstack-nova11:09
*** threestrands has quit IRC11:10
*** ttsiouts has quit IRC11:18
*** nightmare_unreal has joined #openstack-nova11:23
*** slaweq has joined #openstack-nova11:24
*** ociuhandu has joined #openstack-nova11:27
*** ttsiouts has joined #openstack-nova11:31
*** ociuhandu has quit IRC11:32
*** rcernin has joined #openstack-nova11:38
*** rcernin has quit IRC11:43
*** ttsiouts has quit IRC11:46
*** rpittau|bbl is now known as rpittau11:54
*** ratailor has quit IRC12:02
*** slaweq has quit IRC12:13
*** ttsiouts has joined #openstack-nova12:17
*** nweinber has joined #openstack-nova12:18
*** tkajinam has quit IRC12:25
*** ociuhandu has joined #openstack-nova12:32
*** ttsiouts has quit IRC12:32
*** fyx has quit IRC12:32
*** fyx has joined #openstack-nova12:32
*** bcm has quit IRC12:32
*** mnaser has quit IRC12:32
*** jungleboyj has quit IRC12:33
*** gmann has quit IRC12:33
*** lseki has quit IRC12:33
*** nicolasbock has quit IRC12:33
*** jungleboyj has joined #openstack-nova12:33
*** bcm has joined #openstack-nova12:33
*** lseki has joined #openstack-nova12:33
*** gmann has joined #openstack-nova12:33
*** nicolasbock has joined #openstack-nova12:33
*** mnaser has joined #openstack-nova12:33
*** ttsiouts has joined #openstack-nova12:35
*** ociuhandu has quit IRC12:36
*** slaweq has joined #openstack-nova12:43
*** jamesdenton has joined #openstack-nova12:46
*** slaweq has quit IRC12:47
artomstephenfin, don't you have the hackfest?12:48
artomIs that in the next two weeks?12:48
stephenfinend of the month12:48
stephenfinso all good12:49
stephenfinAnyone tried deploying devstack on an RPi?12:49
artomstephenfin, ah, sorry, in my head they were back to back12:51
*** mriedem has joined #openstack-nova12:55
*** bbowen has joined #openstack-nova13:15
*** owalsh_ is now known as owalsh13:20
*** efried has joined #openstack-nova13:20
*** boxiang has quit IRC13:22
*** boxiang has joined #openstack-nova13:22
*** ttsiouts has quit IRC13:31
*** lbragstad has joined #openstack-nova13:34
*** jmlowe has joined #openstack-nova13:38
*** jaosorior has quit IRC13:44
*** sangeet has joined #openstack-nova13:52
openstackgerritDan Smith proposed openstack/nova master: DNM: Try to make a glance multistore job  https://review.opendev.org/73418413:53
dansmithsean-k-mooney: yeah, default is uwsgi and apparently none of the magic glance stuff works without mod_wsgi.. the image import, image conversion, etc13:55
sean-k-mooneyah i see13:56
sean-k-mooneythat is likely a glance bug...13:56
dansmithI needed it for the copy-to-store, but forgot to copy that from my local config, but hadn't gotten far enough to notice and it's required for the conversion too13:56
dansmithI dunno, don't we have problems with uwsgi and background tasks?13:56
dansmithit's because those things happen in threads13:56
sean-k-mooneyoh so its the same issue as we have with the heartbeats13:57
dansmithright13:57
sean-k-mooneyi guess you could argure that they are relying on behavior of the implemeation that is not requried13:57
sean-k-mooneybut ya fair point13:57
sean-k-mooneye.g. the lifetime smeantics of mod_wsgi or the python one13:58
dansmithwell, it has been fine for multiple things (pre-wsgi, mod_wsgi) until uwsgi, but yeah they could use a conductor like service13:58
dansmithbut that kinda sucks13:58
dansmithand needs to be on the same machine as the api I think because of the file uploads it needs to access13:58
sean-k-mooneyya fair enough13:59
dansmithanyway, it'll be a dependency for using this stuff13:59
*** ociuhandu has joined #openstack-nova14:05
dansmithsean-k-mooney: so, I'm tweaking my spec to remove the "functional tests should be easy" section, which I've realized is actually going to be super difficult14:13
dansmithbecause we stub out ALL of the image backend stuff in functional tests for libvirt,14:13
sean-k-mooneyi assume we are just going to keep your new multistore ceph job ya14:13
dansmithand would require me to write at least 10x the actual code to just have functional support14:13
dansmithinstead,14:13
sean-k-mooneyif so i think that is more then enough14:13
dansmithI'm thinking we just change the existing job to do what I have here, which means the first tempest test kicks a copy-to-store from the file backend, and then the rest of the tests just have it set as it would14:14
dansmithyeah, cool14:14
dansmithmy only concern is what happens if/when we test snapshot with the file backend as the default,14:15
sean-k-mooneyif we have a regression in the future i think we can corss the function test bridge then14:15
dansmithbut if that does something stupid I might need to do some devstack surgery to make the rbd still the default, but upload the cirros image to the file backend14:15
sean-k-mooneywell i think we can specify the store on upload14:15
dansmithyeah, we can,14:16
dansmithjust requires devstack surgery14:16
dansmithbut I want to see if it just does the smarter thing14:16
sean-k-mooneyyou are setting the default to cheap14:16
sean-k-mooneywhcih is the file backend right14:16
sean-k-mooneybut novas to robust14:16
sean-k-mooneyso the glacce upload shoudl be to file14:16
sean-k-mooneyand then we will use ceph when doing normal snapshots right14:17
dansmithright, nova doesn't specify one when it does the snapshot.. however, since it's rbd-backed, I think snapshot does the ceph stuff under the covers, so it may work just by virtue of the fact that it doesn't actually upload, but just tells glance about it14:17
dansmiththe cirros upload will go to file currently, yeah14:17
sean-k-mooneyis this job using the rbd image backend14:18
dansmithyeah, that's the whole point :)14:18
sean-k-mooneyor are we only using ceph for bfv14:18
sean-k-mooneyya that is what i assuemed just checking14:18
dansmithno, this is rbd on the image backend14:18
*** iurygregory_ has joined #openstack-nova14:19
*** iurygregory has quit IRC14:19
*** iurygregory_ is now known as iurygregory14:19
sean-k-mooneyya so im wondering if we want to either add a post playbook to test the file snapshot or if we want to add multi store to another job for that case14:19
*** udesale_ has joined #openstack-nova14:19
sean-k-mooneyfor example we have a multi cell job right14:20
sean-k-mooneywe could configure multisore on that too and check it uploads to the cell local one14:20
*** jawad_axd has quit IRC14:20
openstackgerritDan Smith proposed openstack/nova master: DNM: Try to make a glance multistore job  https://review.opendev.org/73418414:21
*** udesale has quit IRC14:22
*** ttsiouts has joined #openstack-nova14:23
sean-k-mooneydansmith: by the way before aarents  upload a new version were you expecting to reuse this config option or add a new one https://review.opendev.org/#/c/734776/2/nova/conf/compute.py@91214:26
dansmithsean-k-mooney: tbh, I didn't know we had that one14:26
dansmithI'm not really sure why we have that, as image downloads would be gated by the build limit14:27
sean-k-mooneyso when i asked about it in the ptg i tought we siad it was only for the filter schdulier btu i guess there is a seperate one for compute14:27
*** sapd1 has joined #openstack-nova14:28
dansmiththat config you linked isn't for the filter/weigher, but for compute presumably yeah?14:28
sean-k-mooneycorrect its in compute14:28
sean-k-mooneybut do we use it anywhere https://github.com/openstack/nova/search?q=max_concurrent_disk_ops&unscoped_q=max_concurrent_disk_ops14:29
sean-k-mooneyit was added by https://github.com/openstack/nova/commit/728f20e8f4ac2e3d4b893b7169b81d20471d0be914:29
dansmithin libvirt14:29
sean-k-mooneyah we do  with compute_utils.disk_ops_semaphore14:30
dansmithI mean... I was expecting there to be one for snapshotting as a whole, so we didn't even start a high-level snapshot operation until we could get to it,14:31
dansmiththis would presumably lock much further down the chain, so you're already paused and waiting for room14:31
*** mlavalle has joined #openstack-nova14:31
dansmithwhich seems less than ideal to me14:32
sean-k-mooneyya it looks like its only locking on the actull io operation i.e. invoking qemu-image or glance14:32
*** sapd1 has quit IRC14:33
*** ttsiouts has quit IRC14:34
dansmithright14:34
dansmithI commented14:34
*** _mlavalle_1 has joined #openstack-nova14:34
*** sapd1 has joined #openstack-nova14:34
*** ociuhandu has quit IRC14:36
sean-k-mooneydansmith: that current semaphore is used in snapshot but only at the end https://opendev.org/openstack/nova/src/branch/master/nova/virt/libvirt/driver.py#L246514:36
*** mlavalle has quit IRC14:37
dansmithah for the actual upload I guess?14:37
*** _mlavalle_1 has quit IRC14:37
dansmithseems like that semaphore was added kinda willy-nilly in a bunch of places without a systemic view of "never call this code without the semaphore held"14:37
dansmithbecause I assume if we grab it too high we'll double acquire14:37
stephenfinsean-k-mooney: In brief, what would happen if you configured disk_allocation_ratio too high? Would you see outright corruption of existing images or simply failure to e.g. download images for a new instance?14:37
stephenfinI'm not actually sure what happens with sparse image if you run out of disk space14:38
stephenfin*a14:38
dansmithstephenfin: you'll get disk errors in the guest when you run out of space14:38
dansmithstephenfin: like your disk was failing if it was a real disk14:38
stephenfinand if I tried to schedule a new guest to that host?14:39
sean-k-mooneydansmith: ya probably14:39
dansmithwhich generally means "you asked me to write this to disk, disk said no" which if it's in the journal flush could mean whole fs corruption, but likely a remount-readonly at minimum and application data loss14:39
sean-k-mooneyyou can allso get into cases where you cant log in or exectue commands14:40
sean-k-mooneyin the guest14:40
dansmithfor sure14:40
stephenfindansmith++ sweet, thanks14:40
sean-k-mooneyi guess on the host too if the nova instnace directoy is in /14:40
stephenfinI'm writing an overcommit doc atm and trying to tease out implications of these things14:40
sean-k-mooneyor under it on the same partion14:40
dansmithsean-k-mooney: yes, it could be quite bad14:41
sean-k-mooneystephenfin: honestly disk allocation ratio shoudl have a max of 114:41
dansmithsean-k-mooney: presumably the host fs has some reserved-for-root set and qemu can't exhaust that though14:41
sean-k-mooneydansmith: you would hope.14:41
dansmithsean-k-mooney: agree, overcommit of disk is just asking for problems14:41
sean-k-mooneypeopel get a way with it with lvm thin providiouning by carfully monitoring it14:42
sean-k-mooneybut it makes me uncomfortable if i care about the data14:42
dansmithit's the worst thing you can overcommit14:42
sean-k-mooneydansmith: the host_reserved_disk_mb helps but if you allow suspend you can exceed that as we need to snapshot the guest ram and save it to disk14:43
sean-k-mooneywhen kolla was young it did nto have logrotate and i made the mistake of running our internal intel dev cluserter with debug logging on14:44
sean-k-mooneyfixing out of disk was fun...14:44
sean-k-mooneybecause it caused db curruption14:44
dansmithI think stephenfin gets the picture.. "it's bad, yo"14:44
sean-k-mooneystephenfin: are you going to write doc to cover the interaction betweeen file backed memory and reserved disk by the way14:45
sean-k-mooneyif not i can proably try to write something14:46
stephenfinyou mean above what I put in the config opt docs here https://review.opendev.org/#/c/734658/ ?14:46
stephenfinoh, reserved _disk_14:46
sean-k-mooneyyes14:46
sean-k-mooneydisk14:46
sean-k-mooneyand i guess we shoudl menetion the reserved hugepages option too14:47
stephenfinhmm, I hadn't thought of that. Only seeing your comments now14:47
stephenfinSure, I can do that early next week14:47
sean-k-mooneyok if you dont get to it remind me and i can try to do it14:48
sean-k-mooneybasically if its the same file system they shoudl add the value of the file backed memory option to reserved disk14:48
*** slaweq has joined #openstack-nova14:52
*** factor has quit IRC14:52
*** jraju__ has quit IRC14:57
*** boxiang has quit IRC15:06
*** boxiang has joined #openstack-nova15:07
*** jdillaman has joined #openstack-nova15:13
aarentsstephenfin: if host disk is full qemu may PAUSE himself (by default I think) see  man qemu werror=action,rerror=action)15:15
*** mlavalle has joined #openstack-nova15:15
sean-k-mooneydansmith: looks like glance started listening on http https://zuul.opendev.org/t/openstack/build/8afb850f7844450389823d7f84b90c61/log/controller/logs/screen-g-api.txt#345 but devstack seams to be checking https https://zuul.opendev.org/t/openstack/build/8afb850f7844450389823d7f84b90c61/log/controller/logs/devstacklog.txt#1999515:16
openstackgerritStephen Finucane proposed openstack/nova master: docs: Add a guide on overcommit  https://review.opendev.org/73517515:16
openstackgerritStephen Finucane proposed openstack/nova master: tox: Stop linting as part of docs target  https://review.opendev.org/73517615:16
dansmiththat wsgi thing works for me locally15:17
stephenfinsean-k-mooney, dansmith, aarents: that's the doc anyway15:17
sean-k-mooneydansmith: i can deploy with the local.conf that was generated and see what happens15:17
*** dklyle has joined #openstack-nova15:17
dansmithsean-k-mooney: it's getting a 503, which means it's hitting apache or something which should be proxying15:17
sean-k-mooneyim wondering if it enabling the tls code15:17
sean-k-mooneye.g. the tls proxy15:18
sean-k-mooneybut not actully configuring it for some reason15:18
dansmithidk15:18
*** hamalq has joined #openstack-nova15:18
sean-k-mooneythe tls proxy is enabled15:18
sean-k-mooneyhttps://94b38a054383d32ffbe4-eecb7ff79b5c6e121d6c20fabad8af79.ssl.cf1.rackcdn.com/734184/10/check/nova-ceph-multistore/8afb850/controller/logs/local_conf.txt15:18
dansmithunless something changed recently15:19
dansmithmaybe the ceph plugin job I'm inheriting from does that?15:19
*** hamalq has quit IRC15:19
dansmithbecause at least a couple weeks ago, devstack didn't with just that wsgi switch in the localrc15:19
sean-k-mooneydansmith: i think its enabeld by default15:19
dansmithin d-g? because it wasn't for me, unless it's changed very recently15:20
sean-k-mooneyin the devstack or tempest job15:20
*** hamalq has joined #openstack-nova15:20
sean-k-mooneynot d-g but the base devstack zuulv3 job i think15:20
*** slaweq has quit IRC15:20
dansmithyeah, that's what I mean15:20
dansmithsomething other than devstack defaults15:20
dansmithwell, I'd not think this would change it, unless the base devstack job breaks with wsgi15:21
dansmithmaybe glance has a mod_wsgi job I can look at15:21
dansmithdoesn't look like it15:21
sean-k-mooneywhich raiase the question how do they test this?15:22
aarentssean-k-mooney: dansmith Should I need a reno for a new config parameter ?15:22
sean-k-mooneylooking at there zuul jobs they only use the normal tepest jobs15:22
dansmith2020-06-11 14:59:39.532624 [proxy:error] [pid 15204:tid 140023944947456] (111)Connection refused: AH00957: HTTP: attempt to connect to 10.0.1.13:19292 (10.0.1.13) failed15:22
dansmithsean-k-mooney: yeah ....15:23
sean-k-mooneydansmith: am its runing on a different port15:23
dansmithyep15:23
sean-k-mooney  wsgi starting up on http://0.0.0.0:1919115:23
sean-k-mooneyso close15:24
dansmithGLANCE_URL is different for uwsgi15:24
dansmithso maybe I need to set that15:24
sean-k-mooneyproably arent you having fun working on glance things...15:24
dansmithor just change the service port15:24
dansmithno :)15:24
dansmithhmm,15:25
dansmith19191 is the registry port supposedly15:25
dansmithand 19292 is the glance_service_port15:25
sangeetI am trying to get some help on turning on TLS for nova. I was able to but nova-api is not interpreting the messages correctly. I had posed this opensatck channel as well but now response. Any help will be much apprecaited. Nova.conf is http://paste.openstack.org/show/79464515:26
sangeet192.168.243.242 - - [11/Jun/2020 14:00:22] code 400, message Bad request syntax ('\x16\x03\x01\x02\x00\x01\x00\x01\xfc\x03\x03O\xafjh.\xe4\xeb\x88\xfa\x9d\xff\x18\x813\x7f0\xa2\xe0\xb9J`\x15f\xb7=)\xf2}td~\x0e\x00\x00n\xc00\xc0,\xc02\xc0.\xc0/\xc0+\xc01\xc0-\x00\xa5\x00\xa3\x00\xa1\x00\x9f\x00\xa4\x00\xa2\x00\xa0\x00\x9e\xc0(\xc0$\xc0\x14\xc0')15:26
sangeetAbove log is from nova-api15:27
sean-k-mooneysangeet: so nova itslef does not support tls you have to use a webserver to terminate the tls session before it hits the nova api15:28
sean-k-mooneythat kind of looks like the api is getting the encyped payload15:29
sangeetThank you sean-k-mooney15:45
gibinova meeting starts in 10 minutes on #openstack-meeting-315:49
*** ircuser-1 has joined #openstack-nova15:50
*** markvoelker has joined #openstack-nova15:54
*** markvoelker has quit IRC15:56
*** belmoreira has quit IRC15:57
*** rpittau is now known as rpittau|afk15:58
jsuchomedansmith: hi, thanks for your latest review, do you want me to reduce that comment about cyclic import or is there anything else?16:00
*** vishalmanchanda has quit IRC16:00
*** gyee has joined #openstack-nova16:03
openstackgerritDan Smith proposed openstack/nova master: DNM: Try to make a glance multistore job  https://review.opendev.org/73418416:05
*** ociuhandu has joined #openstack-nova16:06
openstackgerritDan Smith proposed openstack/nova-specs master: Update the testing section of the rbd-glance-multistore spec  https://review.opendev.org/73519216:13
dansmithjsuchome: personally I think it should be reduced in line with the other examples I quoted16:14
dansmithjsuchome: I don't remember if there was anything else, but I'll have to circle back next week.. trying to get some stuff done before I disappear tomorrow16:15
*** iurygregory has quit IRC16:15
*** ociuhandu has quit IRC16:15
*** sapd1 has quit IRC16:19
*** boxiang has quit IRC16:19
*** boxiang has joined #openstack-nova16:19
sean-k-mooneydansmith: this is the cyborg shelve path fyi https://review.opendev.org/#/c/729563/216:45
bauzasgibi: fwiw, I'd have loved to volunteer for helping you during your holidays, but I'll be off at the same time16:46
gibibauzas: ack, thanks16:46
gibiand don't worry16:47
dansmithsean-k-mooney: ack, will try to keep that in mind in the coming weeks16:57
sean-k-mooneyi would like to see ci test it first so ill update this https://review.opendev.org/#/c/709641/17:01
*** derekh has quit IRC17:01
gibi19:08 -ChanServ(ChanServ@services.)- You are not authorized to (de)op gibi on17:08
gibi          #openstack-nova.17:08
gibiso I need help to update the challen topic to point to the victori runway etherpad17:09
*** udesale_ has quit IRC17:19
*** bbowen has quit IRC17:22
*** artom has quit IRC17:26
*** artom has joined #openstack-nova17:26
sean-k-mooneygibi: i think dansmith has op on this channel?17:30
*** ChanServ sets mode: +o dansmith17:31
* dansmith speaks with a booming voice17:31
*** dansmith changes topic to "Current runways: https://etherpad.openstack.org/p/nova-runways-victoria -- This channel is for Nova development. For support of Nova deployments, please use #openstack."17:31
sean-k-mooneyhehe17:31
dansmithgood?17:31
sean-k-mooneyseams to work for me17:32
sean-k-mooneyand /topic works so yep17:32
gibidansmith: thanks17:34
*** ChanServ sets mode: -o dansmith17:34
openstackgerritDan Smith proposed openstack/nova master: Check cherry-pick hashes in pep8 tox target  https://review.opendev.org/73520517:44
dansmithsean-k-mooney: did you have a stable commit with broken hashes in gerrit that is still yet to be fixed?17:45
dansmithwould like to use it to test17:45
*** mlavalle has quit IRC17:50
*** nightmare_unreal has quit IRC17:51
dansmithsean-k-mooney: ^ ?18:04
sean-k-mooneyam im not sure if stephen fixed it18:05
sean-k-mooneyill check18:05
dansmithokay I wasn't sure if you had proposed it or someone else.. I don't see any recent -1ed patches on stable from you18:05
sean-k-mooneyhttps://review.opendev.org/#/c/728057/ was the one i think18:06
dansmithokay that's not the one I was thiking of but yeah I see18:06
sean-k-mooneyhttps://review.opendev.org/#/c/728059/1 will be broken too18:06
sean-k-mooneysince it has not been update to pick up stephnes fix18:07
sean-k-mooneyalso https://review.opendev.org/#/c/703115/18:08
sean-k-mooneyagain stephen updated it yesterday18:08
dansmithI guess any patch with unmerged cherry picks would technically fail my check18:08
dansmithso maybe that's enough18:08
*** factor has joined #openstack-nova18:08
*** eharney has quit IRC18:09
sean-k-mooneyya basically if  you dont wait for the preveious one to merge then it will be broken more then likely18:09
sean-k-mooneywell i guess only if you are asked to modify things18:09
dansmithor you don't realize it fails for some other reason18:09
sean-k-mooneyya18:09
dansmithmost people don't run tests on their backports, they wait until zuul tells them something is broken18:10
sean-k-mooneyya if i have conflict i run the tests locally i often dont if it cleanly applied18:11
openstackgerritDan Smith proposed openstack/nova stable/ussuri: Check cherry-pick hashes in pep8 tox target  https://review.opendev.org/73520818:11
dansmithyup18:11
sean-k-mooneyit depens on the patch i guess and how much i think its likely to break18:11
dansmithso the backport of my check should fail if it hasn't landed on master yet18:12
sean-k-mooneydo you plan to have each stable job just check the new branch18:13
sean-k-mooneye.g. stien checks it merged on train which in trun check ussuri whcih checks master18:13
dansmithI'm just checking that all referenced hashes exist on either master or stable/* which I think is fine18:14
sean-k-mooneyya ok that should work18:14
*** tesseract has quit IRC18:15
dansmithif you backport something to stable before it's merged on master, zuul will fail it, which would also help us notice that the backport shouldn't be merged yet18:15
sean-k-mooneyso https://review.opendev.org/#/c/735208/1 should fail because https://review.opendev.org/#/c/735205/1 is not merged18:15
dansmithmight be too much for some people, but seems like a good idea to me18:15
dansmithright18:15
sean-k-mooneyya i kind of feel like that should be a standard zuul check18:15
dansmithyeah18:16
dansmithwe'll see if it works18:16
sean-k-mooneyif it does maybe we could extend the merge-check job18:16
sean-k-mooneyok im going to sign off soon. im takeing monday and tuseday off too so ill be back online on wednesday18:17
dansmithcool, enjoy!18:17
openstackgerritDan Smith proposed openstack/nova master: Check cherry-pick hashes in pep8 tox target  https://review.opendev.org/73520518:25
openstackgerritDan Smith proposed openstack/nova stable/ussuri: Check cherry-pick hashes in pep8 tox target  https://review.opendev.org/73520818:27
*** bbowen has joined #openstack-nova18:30
*** sean-k-mooney has quit IRC18:33
*** jsuchome has quit IRC19:02
*** ralonsoh has quit IRC19:15
*** avolkov has quit IRC19:24
gmannelod: i was just reading log for cherry-pick things and the line you mentioned that "if patch on previous branch is not merged then it does not add cherry-pick from". but it does in both case either backported to all branch at once or one by one once merge in previous branch.19:39
elodgmann: so if a patch has merged on master, and then i cherry-pick it to ussuri, and before it gets merged in ussuri i cherry pick it to stable stein *from* ussuri, you say that the patch in stein will have the 'cherry-picked' lines both from master and ussuri?19:43
gmannelod: i think so but what i did previously was. master patch not merged yet and cherry picked to train, stein and all had 'cherry-piked from'..19:44
*** hemna has quit IRC19:44
elod* s/stein/train/19:46
elodhmmm, I'll check it in the coming days, however I remember that it worked as I wrote19:47
elodbut I usually cherry-pick in cli, not via gerrit, so maybe I'm wrong19:48
*** slaweq has joined #openstack-nova19:49
gmannelod: i remember it does via gerrit where i end up having old commit id in 'cheery picked from' when i updated master patch.19:52
elodgmann: I've cherry-picked a not merged patch from rocky to queens and it did not add the cherry-picked line: https://review.opendev.org/#/c/735222/19:53
eloddid it via gerrit ui cherry-pick button19:54
gmannelod: may be doing it from master it does? i did from master.19:56
*** hemna has joined #openstack-nova19:57
elodgmann: yes, that's possible. do you have an example from the past days maybe?19:57
gmannlet me check19:57
gmannelod: this one, commit id in cherry pick from are wrong there :) - https://review.opendev.org/#/q/I96bb9fcb55f2a4fc3aefa01db70b45740db166c819:58
gmanndansmith: how did you test your pep8 check? should we backport it and test with https://review.opendev.org/#/c/728059/ ? or any DNM patch on master and backport to ussuri20:00
dansmithgmann: check the backport of the check -- it's self-checking :)20:01
gmannohk, just saw that20:02
elodgmann: hmm, yes, it seems, if we backport from master, then it adds the cherry-picked line, even if that is not yet merged :S you are right20:02
elodso to avoid this, we have to wait until patch on master gets merged ;)20:04
gmannyeah that is best way and with dansmith proposed pep8 check will be helpful to wait or at least make checky pick from commit id upto date20:05
gmannseems like pdf doc broken again?20:05
*** nweinber has quit IRC20:10
dansmithgmann: yeah seems like it20:11
gmanni think we need to stop -vv option.20:11
openstackgerritJiri Suchomel proposed openstack/nova master: Add ability to download Glance images into the libvirt image cache via RBD  https://review.opendev.org/57430120:13
dansmithgmann: so are you cool with that pep8 backport check?20:19
gmanndansmith: yeah, i am +2. i agree that is helpful.20:20
dansmithsweet, I think so too20:21
dansmithI'm always worried I'm going to approve something on N-2 before N-120:22
gmanni wanted to move that to hacking :) if elod agree from stable backport perspective as general.20:22
dansmithso this helps that, in addition to the hash change problem20:22
gmanntrue and i recently forgot to update the commit id when i updated master one later20:22
dansmithgmann: feel free to move it if you want it somewhere other than where I'm calling it from20:22
dansmithgmann: ack, I'm sure I've done that in the past, despite trying to be good about it20:22
melwittgmann: why do you think -vv is causing the doc build failure? I'm looking at the log and don't yet see why it failed20:23
gmannmelwitt: not that, i was confused with echo message of too verbose :).20:24
melwitthttps://zuul.opendev.org/t/openstack/build/a678c8c3717d48a2aee415b1eea54a8e/log/sphinx-build-pdf.log#760420:24
gmannits different problem. i can see lof of  LaTeX warning but not sure that leads to error20:24
melwittyeah same20:24
*** slaweq has quit IRC20:26
elodgmann: yes, I think it would be good :)20:28
dansmithelod: I guess we could, on stable, fail if we find no cherry-pick lines, unless the "StableOnly" tag is in the commit message20:30
eloddansmith: sounds good!20:31
elodis see e.g. train-only, stein-only, etc tags in commit messages, too, however. maybe those can be accepted next to stable-only as well20:36
gmann+1,  with 'StableOnly' tag it can work fine20:36
gmannjust stable-only is easy20:36
*** gyee has quit IRC20:36
dansmithmaybe StableOnly: train ?20:36
elodI'm OK with that20:36
dansmithI'll just make it grep for StableOnly and we can decide20:37
gmannbut that may be  backported to stein and so.20:37
gmannlike i start fixing from train till rocky20:37
gmanndevstack has such cases very often20:38
elodStableOnly: train is a bit superfluous20:38
elodlet just be it simply stable-only or [stable-only] :)20:38
gmannwe can say 'StableOnly' means not coming from master but it can be backported to older stable20:39
openstackgerritDan Smith proposed openstack/nova master: Check cherry-pick hashes in pep8 tox target  https://review.opendev.org/73520520:40
dansmithgmann: yeah I think that's the way to do it20:40
dansmith"StableOnly means it originated on some stable branch"20:40
gmannyeah20:40
openstackgerritDan Smith proposed openstack/nova stable/ussuri: Check cherry-pick hashes in pep8 tox target  https://review.opendev.org/73520820:41
openstackgerritDan Smith proposed openstack/nova stable/ussuri: DNM: Testing a thing  https://review.opendev.org/73524120:41
*** cz3 is now known as WujaszekKrypta20:43
*** mgariepy has quit IRC20:44
*** WujaszekKrypta is now known as cz320:45
*** hemna has quit IRC20:45
*** mgariepy has joined #openstack-nova20:48
*** artom has quit IRC21:13
*** martinkennelly has quit IRC21:27
*** martinkennelly has joined #openstack-nova21:27
*** JamesBenson has quit IRC21:38
openstackgerritMerged openstack/nova stable/train: Add functional test for bug 1550919  https://review.opendev.org/73127221:48
openstackbug 1550919 in OpenStack Compute (nova) train "[Libvirt]Evacuate fail may cause disk image be deleted" [Medium,In progress] https://launchpad.net/bugs/1550919 - Assigned to Lee Yarwood (lyarwood)21:48
*** rcernin has joined #openstack-nova21:49
*** markvoelker has joined #openstack-nova21:49
*** markvoelker has quit IRC21:54
*** dustinc has joined #openstack-nova21:55
*** martinkennelly has quit IRC21:57
*** martinkennelly has joined #openstack-nova21:58
*** mriedem has left #openstack-nova21:59
*** artom has joined #openstack-nova22:13
openstackgerritDan Smith proposed openstack/nova master: Check cherry-pick hashes in pep8 tox target  https://review.opendev.org/73520522:19
openstackgerritDan Smith proposed openstack/nova stable/ussuri: Check cherry-pick hashes in pep8 tox target  https://review.opendev.org/73520822:19
dansmithelod: gmann: ^ I'm out tomorrow FYI22:19
*** threestrands has joined #openstack-nova22:20
*** tkajinam has joined #openstack-nova22:46
*** martinkennelly has quit IRC22:48
*** markvoelker has joined #openstack-nova22:49
*** markvoelker has quit IRC22:54
*** sangeet has quit IRC22:56
*** tosky has quit IRC23:10
*** hamalq has quit IRC23:20
*** hamalq has joined #openstack-nova23:20
openstackgerritMerged openstack/nova stable/train: libvirt: Don't delete disks on shared storage during evacuate  https://review.opendev.org/73127323:33
*** rcernin has quit IRC23:48
*** hamalq has quit IRC23:50
*** rcernin has joined #openstack-nova23:51

Generated by irclog2html.py 2.17.2 by Marius Gedminas - find it at https://mg.pov.lt/irclog2html/!