Monday, 2021-04-12

« Sunday, 2021-04-11 Index Tuesday, 2021-04-13 »
*** tosky has quit IRC00:00
*** ociuhandu has joined #openstack-nova00:15
*** ociuhandu has quit IRC00:33
*** LinPeiWen has joined #openstack-nova00:39
*** ociuhandu has joined #openstack-nova01:00
*** brinzhang has joined #openstack-nova01:02
*** ociuhandu has quit IRC01:05
*** iurygregory has quit IRC01:05
*** ociuhandu has joined #openstack-nova01:12
*** ociuhandu has quit IRC01:16
*** ociuhandu has joined #openstack-nova01:30
*** ociuhandu has quit IRC01:54
*** ociuhandu has joined #openstack-nova02:22
openstackgerritQiu Fossen proposed openstack/nova-specs master: Repropose copying pmem devices data spec  https://review.opendev.org/c/openstack/nova-specs/+/78556302:41
*** k_mouza has joined #openstack-nova02:44
*** ociuhandu has quit IRC02:45
*** k_mouza has quit IRC02:48
*** zzzeek has quit IRC02:53
*** zzzeek has joined #openstack-nova02:55
*** ociuhandu has joined #openstack-nova03:27
*** ociuhandu has quit IRC03:44
*** brinzhang_ has joined #openstack-nova04:00
*** brinzhang has quit IRC04:03
*** eharney has quit IRC04:11
*** tristanC_ has joined #openstack-nova04:15
*** jrosser has quit IRC04:20
*** tristanC has quit IRC04:20
*** mjturek has quit IRC04:20
*** eharney has joined #openstack-nova04:30
*** jrosser has joined #openstack-nova04:34
*** vishalmanchanda has joined #openstack-nova04:41
*** ricolin has joined #openstack-nova04:41
*** ociuhandu has joined #openstack-nova04:56
*** ociuhandu has quit IRC05:00
*** ociuhandu has joined #openstack-nova05:12
*** ociuhandu has quit IRC05:16
*** ociuhandu has joined #openstack-nova05:28
*** ociuhandu has quit IRC05:32
*** whoami-rajat_ has joined #openstack-nova05:36
*** ociuhandu has joined #openstack-nova05:38
*** sapd1 has joined #openstack-nova05:51
*** ociuhandu has quit IRC05:51
*** ociuhandu has joined #openstack-nova05:52
*** ociuhandu has quit IRC05:56
*** sapd1 has quit IRC05:59
*** ralonsoh has joined #openstack-nova06:02
*** slaweq has joined #openstack-nova06:09
*** hoonetorg has quit IRC06:21
*** ociuhandu has joined #openstack-nova06:22
*** ociuhandu has quit IRC06:30
*** luksky has joined #openstack-nova07:01
*** ociuhandu has joined #openstack-nova07:07
*** ociuhandu has quit IRC07:07
*** andrewbonney has joined #openstack-nova07:08
*** xinranwang has joined #openstack-nova07:29
*** tosky has joined #openstack-nova07:35
*** Luzi has joined #openstack-nova07:42
bauzasgood morning Nova07:47
*** lucasagomes has joined #openstack-nova07:59
*** rpittau|afk is now known as rpittau08:04
*** links has joined #openstack-nova08:07
*** ociuhandu has joined #openstack-nova08:09
*** martinkennelly has joined #openstack-nova08:19
*** k-s-dean has joined #openstack-nova08:26
gibibauzas: good morning08:29
bauzaseventually freed from PTOs and kids08:31
yonglihegood morning08:33
yonglihebauzas, free from kids, really, tell me how08:34
bauzasyonglihe: by passing them to the grandparents during the vacations ;)08:35
yonglihe-:)08:35
bauzas(during the *school* vacations tbc=08:35
openstackgerritLucas Alvares Gomes proposed openstack/nova master: [OVN] Explicitly set nova-next job to ML2/OVS  https://review.opendev.org/c/openstack/nova/+/77694408:37
*** derekh has joined #openstack-nova08:39
xinranwanggibi:  Hi gibi, I have reproposed the smartnic spec, please review it when you got time.  thanks! https://review.opendev.org/c/openstack/nova-specs/+/78363208:40
openstackgerritLucas Alvares Gomes proposed openstack/nova master: [OVN] Explicitly set grenade job to ML2/OVS  https://review.opendev.org/c/openstack/nova/+/77693408:40
openstackgerritLucas Alvares Gomes proposed openstack/nova master: [OVN] Explicitly set nova-next job to ML2/OVS  https://review.opendev.org/c/openstack/nova/+/77694408:40
rpittaugood morning everyone! Not sure if it's a known issue, when running grenaed jobs in stable/victoria in ironic, we're seeing this error in nova-api loga: http://paste.openstack.org/show/804380/08:46
rpittauthis is an example of  failing job: https://zuul.opendev.org/t/openstack/build/90de727c38d54f4b80f6998ff3b19c1708:47
bauzasrpittau: looking08:51
rpittauthanks!08:51
bauzaslooks like an oslo messaging version issue08:53
bauzaseg. https://zuul.opendev.org/t/openstack/build/90de727c38d54f4b80f6998ff3b19c17/log/controller/logs/screen-n-api.txt#203408:53
bauzasbut that's on the old ussuri API08:54
rpittauinteresting, we started noticing the failure just last friday08:54
openstackgerritQiu Fossen proposed openstack/nova-specs master: Allow migrating PMEM's data  https://review.opendev.org/c/openstack/nova-specs/+/78556308:55
*** brinzhang_ is now known as brinzhang08:55
*** dtantsur|afk is now known as dtantsur08:56
openstackgerritElod Illes proposed openstack/nova stable/train: DNM: nova-grenade-multinode test  https://review.opendev.org/c/openstack/nova/+/78573108:56
bauzasrpittau: list of installed packages https://zuul.opendev.org/t/openstack/build/90de727c38d54f4b80f6998ff3b19c17/log/job-output.txt#5128608:57
openstackgerritQiu Fossen proposed openstack/nova-specs master: Allow migrating PMEM's data  https://review.opendev.org/c/openstack/nova-specs/+/78556308:57
bauzasI'm not a grenade specialist, but it looks like it happens with the original install, before upgrading, while testing the original deployment08:58
openstackgerritElod Illes proposed openstack/nova stable/train: DNM: nova-grenade-multinode test  https://review.opendev.org/c/openstack/nova/+/78573108:58
*** ociuhandu has quit IRC08:58
rpittaubauzas: yeah, I can confirm as the job fails immediately08:58
bauzaselod: have you seen any issues with grenade on stable/victoria jobs ?08:59
*** ociuhandu has joined #openstack-nova08:59
rpittauso nova doesn't like the version of oslo messaging installed in ussuri08:59
bauzasrpittau: that's what I'm trying to see08:59
rpittauok, thanks08:59
*** k_mouza has joined #openstack-nova09:03
*** ociuhandu has quit IRC09:09
elodbauzas: no, not in victoria. stable/train is broken though (nova-grenade-multinode), but hopefully fixed by this patch when it merges: https://review.opendev.org/c/openstack/grenade/+/78583109:13
sean-k-mooneyhttps://zuul.opendev.org/t/openstack/build/90de727c38d54f4b80f6998ff3b19c17/log/controller/logs/pip3-freeze.txt#10109:13
bauzaselod: looks like it could be related with rpittau09:13
sean-k-mooneyoslo.messaging==12.5.209:13
bauzasthey don't correctly pin the versions IMHO09:14
bauzassean-k-mooney: yup09:14
sean-k-mooneythe version would be pinned in UC09:14
sean-k-mooneyright?09:15
sean-k-mooneyhttps://github.com/openstack/requirements/blob/stable/ussuri/upper-constraints.txt#L274-L27709:15
sean-k-mooneyso that is above the UC constrait for ussuri09:15
sean-k-mooneyit is the correct one for victoria09:16
sean-k-mooneybut grenade has not run yes09:16
sean-k-mooney*yet09:16
sean-k-mooneymaybe it has. th ejob logs look weird09:17
sean-k-mooneyi dont see the devstack or grenade log09:17
sean-k-mooneywell the devstack log is in old09:18
bauzasI'm looking at https://zuul.opendev.org/t/openstack/build/90de727c38d54f4b80f6998ff3b19c17/log/job-output.txt09:18
bauzasand I can see the upgrade09:18
elodbauzas rpittau : at a first glance the failure looks different. anyway, there was a change in grenade on last Thursday, maybe related: https://review.opendev.org/q/topic:"standardize-tempest-tox-constraints"+status:merged09:18
sean-k-mooneyactully the devstack log has the grenade output in it?09:18
sean-k-mooneyhttps://zuul.opendev.org/t/openstack/build/90de727c38d54f4b80f6998ff3b19c17/log/controller/logs/old/devstacklog.txt09:19
sean-k-mooneyif this is the new zuul v3 grenade job it looks like its overriding the devstack log with the greneade log09:21
sean-k-mooneyit looks like LOGFILE="/opt/stack/old/logs/devstacklog.txt" is not working properly09:25
sean-k-mooneythe devstack logs are just in the main output file09:25
*** ociuhandu has joined #openstack-nova09:30
bauzashonestly, I'm damn puzzled09:33
elodthis looks also interesting: https://zuul.opendev.org/t/openstack/build/90de727c38d54f4b80f6998ff3b19c17/log/job-output.txt#48860-4886409:34
*** ociuhandu has quit IRC09:35
elodthat shows some error, but wierdly the end it succeeds09:35
sean-k-mooneythis is the old resolver09:35
sean-k-mooneyand its waring about the new one right09:36
*** ociuhandu has joined #openstack-nova09:37
rpittauelod: thans for pointing to that change, maybe asking gmann could be worth it09:37
sean-k-mooneyhttps://zuul.opendev.org/t/openstack/build/90de727c38d54f4b80f6998ff3b19c17/log/job-output.txt#4845509:37
sean-k-mooneyinteresting so it install the correct verion and here it upgrade beyond the correct version09:37
sean-k-mooneywell its the victoria version09:38
sean-k-mooneyhum i guess that is after devstack completed initally09:40
sean-k-mooneyelod: bauzas  is grenade ment to be enabled in the first run?09:41
sean-k-mooneyit look like devstack finsihedn and then it imediatly started runnign grenade without any tempest test in between09:41
*** ociuhandu has quit IRC09:43
*** ociuhandu has joined #openstack-nova09:43
sean-k-mooneyelod: bauzas  what happening is its doing a normal devstack install and then its upgrading ironic only and then running tempest09:45
elodthere should be a smoke test. at least in nova's grenade jobs there is09:45
sean-k-mooneyright that is what i expect09:45
sean-k-mooneybut for some reason this is going strait to the upgrade of ironic and does not upgrade other services and then runs tempest09:45
sean-k-mooneyso upgrading ironic pulls in the victoria verion of oslo.messaging which is incompatible with ussui of everything else09:46
sean-k-mooneyactully its upgrading ironic then tempest nou tnot the rest09:47
sean-k-mooneyelod: the smoke tests are disabled https://zuul.opendev.org/t/openstack/build/90de727c38d54f4b80f6998ff3b19c17/log/controller/logs/grenade_localrc.txt#1609:55
sean-k-mooneyfor both the pre and post upgrade09:55
sean-k-mooneyit does not look like UPGRADE_PROJECTS is correct https://github.com/openstack/grenade/blob/stable/ussuri/grenade.sh#L361-L36509:56
sean-k-mooneyor rather its not upgrading all the porjects unless i missed something09:56
sean-k-mooneyits just upgrading ironic09:56
sean-k-mooneythen it upgrades tempest09:56
*** k_mouza has quit IRC09:57
sean-k-mooneyhttps://zuul.opendev.org/t/openstack/build/90de727c38d54f4b80f6998ff3b19c17/log/job-output.txt#47896-4789709:58
sean-k-mooneyit looks like plugins are registered for upgrade10:00
sean-k-mooneyhttps://github.com/openstack/grenade/blob/08199218df99ead17d7c34256ed0969b8ae3b3af/inc/upgrade#L101-L11110:00
elodok, so it works a bit different then in nova10:00
rpittausean-k-mooney, elod, bauzas, it seems fixed by https://review.opendev.org/q/If5f14654ab9aee2a140bbfb869b50d63cb289fdf10:01
sean-k-mooneyin that it does not run test before upgrading yes10:01
rpittauplease check #openstack-qa10:01
bauzasack10:01
sean-k-mooneyrpittau: that is not the fix10:01
rpittaummm wrong link?10:01
sean-k-mooneyat least i dont think so10:01
sean-k-mooneyi think the package is being installed in the ironic upgrade10:02
rpittausean-k-mooney: according to yoctozepto it is10:02
sean-k-mooneynot in the tempest one10:02
rpittauok10:02
sean-k-mooneyya its form 2021-04-09 22:14:21.761580 | controller | 2021-04-09 22:14:21.761 | + inc/python:setup_package:445             :   pip_install -e /opt/stack/new/ironic10:03
sean-k-mooneythat pulls in Collecting oslo.messaging===12.5.210:03
sean-k-mooneyrpittau: the issue here is the ironic job is only upgrading ironic and tempest10:04
sean-k-mooneyrpittau: it need to upgrade all installed services10:04
sean-k-mooneybecause they all share the same version of oslo.messaging10:04
yoctozeptorpittau, sean-k-mooney: that thing is fixing, e.g., ironic-grenade-dsvm job10:04
rpittausean-k-mooney: mmm ok I see, I think this was intended for some reason, need to double-check with dtantsur and TheJulia10:04
yoctozepto(on train)10:05
sean-k-mooneyand the one form victorica that ironic is installing when its upgraded breaks ussuri10:05
yoctozeptothe failure was in smoke tests BEFORE upgrade10:05
openstackgerritLee Yarwood proposed openstack/nova stable/ussuri: Add regression test for bug #1908075  https://review.opendev.org/c/openstack/nova/+/78365510:05
openstackbug 1908075 in OpenStack Compute (nova) "Nova allows a non-multiattach volume to be attached to multiple instances *if* its volume state is reset by an admin" [Low,In progress] https://launchpad.net/bugs/1908075 - Assigned to Lee Yarwood (lyarwood)10:05
openstackgerritLee Yarwood proposed openstack/nova stable/ussuri: api: Reject volume attach requests when an active bdm exists  https://review.opendev.org/c/openstack/nova/+/78365610:05
sean-k-mooneyyoctozepto: the smoke tests are disabled10:05
sean-k-mooneyyoctozepto: before and after upgrade in this job10:06
sean-k-mooneyyoctozepto: this job is failing because the victoria version of oslo messagin is been install when ironic is upgraded10:06
sean-k-mooneyyoctozepto: but the rest of openstack is not upgraded10:06
yoctozeptowhat job are we talking about? :D I am only about ironic-grenade-dsvm on train10:06
*** k_mouza has joined #openstack-nova10:06
sean-k-mooneyyoctozepto: ironic-grenade10:07
sean-k-mooneyno -dvsm10:07
yoctozeptooh, the new one; then it's irrelevant as you said10:07
yoctozeptomust be other issue10:07
sean-k-mooneyyep the issue is as i said10:07
sean-k-mooneythe ironic plugin was the only thing run in the upgrade10:07
sean-k-mooneyso its currently trying to run victoria ironic with ussuri everything else but oslo move or removed a decorator10:08
sean-k-mooneyso the ussuri code cant run with the new verion of the oslo package form victoria10:08
sean-k-mooneythe fix is just to upgrade all the porjects10:08
sean-k-mooneyyoctozepto: https://zuul.opendev.org/t/openstack/build/90de727c38d54f4b80f6998ff3b19c17/log/job-output.txt#47896-47897 see it just ran ironic10:09
yoctozeptoyes, I confirm what sean-k-mooney is saying10:12
yoctozeptosorry for confusion, I misread the question10:12
rpittausean-k-mooney, yoctozepto, thanks, I will need to verify with other ironic cores10:12
*** ociuhandu has quit IRC10:13
sean-k-mooneyrpittau: unless ironic is installed in a venv its unsafe to upgrade it in isolation10:13
yoctozepto++10:13
sean-k-mooneyrpittau: since we share depencies10:13
rpittauofc, makes total sense10:13
sean-k-mooneyrpittau: you could try to do that if you use the old UC10:13
*** ociuhandu has joined #openstack-nova10:13
yoctozepto(psst, you can always base your gate on kolla, just saying ;-) )10:14
rpittau:)10:14
rpittauI guess I have a topic for today meeting :)10:14
sean-k-mooneyyoctozepto: do you have full tempest/upgrade jobs in kolla yet?10:14
sean-k-mooneyi know it was a goal a long time ago just never followed up to see if ye completed it10:15
yoctozeptosean-k-mooney: tempest not :-( but last cycle I made it able to track branches of other projects with zuul so that depends-on works etc.10:16
yoctozeptoI got bifrost to check against kolla-ansible10:17
yoctozeptobecause they often broke us :D10:17
sean-k-mooneyyoctozepto: i still feel a little bad that i never had time to do phase 2 of the bifrost work10:17
sean-k-mooneywhen i implemeted the inital work i had planned to decomose it into multiple contaier runnign the differen services instead of using systemd.10:18
sean-k-mooneyoh the other had it worked so :)10:18
yoctozeptosean-k-mooney: https://github.com/markgoddard/bikolla might interest you then10:19
sean-k-mooneyhaha why not just use kolla ironic support10:19
*** __ministry has quit IRC10:19
sean-k-mooneye.g. just enable the minium service for ironic10:19
sean-k-mooneyi assume that is what that does10:20
sean-k-mooneyhum yes it looks like that is correct more or less https://github.com/markgoddard/bikolla/blob/master/etc/kolla/globals.yml10:20
yoctozeptoyes ;-)10:21
yoctozeptojust need to orchestrate the thing10:21
sean-k-mooneyim debating if im going to keep ironic or not when i redeploy my home cluster10:22
sean-k-mooneypart of me wants too but i use it rarely10:22
sean-k-mooneyand it seams to have issue with ubunutu10:22
sean-k-mooneymy centos image work fine10:22
sean-k-mooneybut i dont think the ipa image or cloud init fully work with ubunuts netplan conrigurtion10:23
sean-k-mooneyso i end up installing the os with a USB when i need anything other then centos anyway10:23
*** k_mouza has quit IRC10:24
*** k_mouza has joined #openstack-nova10:24
*** __ministry has joined #openstack-nova10:25
*** whoami-rajat_ is now known as whoami-rajat10:33
sean-k-mooneystephenfin: im getting pinged for https://review.opendev.org/q/topic:%22bug%252F1910466%22+(status:open%20OR%20status:merged) downstream in bz. when you have time can you rereview those10:34
dtantsurrpittau, sean-k-mooney, grenade should update all services.. breakage in infra jobs?10:34
stephenfinsean-k-mooney: sure10:34
dtantsurmaybe we did something wrong when migrating to zuul v3?10:34
sean-k-mooneydtantsur: perhaps only the plugins seam to be listed10:34
*** k_mouza_ has joined #openstack-nova10:35
*** k_mouza_ has quit IRC10:35
*** k_mouza_ has joined #openstack-nova10:36
elodjust for the record I've looked into the last successful run of ironic-grenade on stable/victoria and it also upgraded to oslo.messaging===12.5.210:37
sean-k-mooneyelod: did it upgrade the other serivces?10:37
sean-k-mooneyits likely not oslo.messaging by the way10:38
sean-k-mooneyi think the reraise decorator comes form oslo.utils10:38
elodsean-k-mooney: yes: Upgrade projects:  keystone placement swift glance neutron nova ironic10:38
*** k_mouza has quit IRC10:39
sean-k-mooneyhttps://github.com/openstack/oslo.utils/blob/master/oslo_utils/excutils.py#L14510:39
sean-k-mooneyelod: ya so thats what has changed10:39
elodsean-k-mooney: it seems10:39
sean-k-mooneynow its just ironic10:39
elodyes10:39
openstackgerritLee Yarwood proposed openstack/nova stable/train: Add regression test for bug #1908075  https://review.opendev.org/c/openstack/nova/+/78584710:40
openstackbug 1908075 in OpenStack Compute (nova) "Nova allows a non-multiattach volume to be attached to multiple instances *if* its volume state is reset by an admin" [Low,In progress] https://launchpad.net/bugs/1908075 - Assigned to Lee Yarwood (lyarwood)10:40
openstackgerritLee Yarwood proposed openstack/nova stable/train: api: Reject volume attach requests when an active bdm exists  https://review.opendev.org/c/openstack/nova/+/78584810:40
sean-k-mooneydo we use the victoria or ussuri version of grenade10:40
sean-k-mooneye.g. is it the source or target version that does the upgrade10:40
elodit should be victoria10:41
elodin my understanding10:41
sean-k-mooneyok then i should be looking at stable/victoria grenade so10:41
sean-k-mooneyi suspect the issue is with the grenaderc generation10:49
sean-k-mooneyUPGRADE_PROJECTS is empty by default10:50
sean-k-mooneyplugins are auto added to it10:50
sean-k-mooneywhich is why ironic is there10:50
sean-k-mooneythere are added by register_project_for_upgrade10:51
sean-k-mooneyhttps://opendev.org/openstack/grenade/src/branch/master/inc/upgrade#L101-L11110:51
*** macz_ has joined #openstack-nova10:53
sean-k-mooneyeach of the in tree services should register them selves https://opendev.org/openstack/grenade/src/commit/15e02fec78cdcce00d302fc2c6c29afea91f6f14/projects/10_keystone/settings#L110:54
sean-k-mooneybut i am wondering if that setting file is correctly loaded10:54
*** macz_ has quit IRC10:57
*** tesseract has joined #openstack-nova11:04
*** brinzhang_ has joined #openstack-nova11:05
yoctozeptook, so still no consensus on what has failed?11:08
yoctozeptooh, I see the missing projects were confirmed11:08
*** brinzhang has quit IRC11:08
yoctozeptoso then it's "only" a matter of tracking what introduced the breaking change in behaviour11:09
*** iurygregory has joined #openstack-nova11:11
sean-k-mooneyyoctozepto: something cause the list of project to be changed ya im not seeing what that was at first glance11:11
yoctozeptosean-k-mooney: ack11:11
*** artom has joined #openstack-nova11:19
*** ociuhandu has quit IRC11:20
*** k_mouza_ has quit IRC11:24
*** k_mouza has joined #openstack-nova11:24
*** k_mouza has quit IRC11:34
*** k_mouza has joined #openstack-nova11:34
*** k_mouza has quit IRC11:37
*** k_mouza has joined #openstack-nova11:37
*** ociuhandu has joined #openstack-nova11:51
*** sapd1 has joined #openstack-nova11:55
*** ratailor has joined #openstack-nova11:55
*** ociuhandu has quit IRC11:58
openstackgerritQiu Fossen proposed openstack/nova-specs master: Support live migrate vtpm server  https://review.opendev.org/c/openstack/nova-specs/+/78586011:59
openstackgerritQiu Fossen proposed openstack/nova-specs master: Allow migrating PMEM's data  https://review.opendev.org/c/openstack/nova-specs/+/78556312:12
*** ociuhandu has joined #openstack-nova12:12
sean-k-mooneystephenfin: didnt you support vtpm live migration already ^12:15
*** ociuhandu has quit IRC12:17
openstackgerritQiu Fossen proposed openstack/nova-specs master: Support live migrate vtpm server  https://review.opendev.org/c/openstack/nova-specs/+/78586012:17
*** mjturek1 has joined #openstack-nova12:17
*** ociuhandu has joined #openstack-nova12:18
*** ociuhandu has quit IRC12:24
*** ratailor has quit IRC12:35
*** ociuhandu has joined #openstack-nova12:38
*** mjturek1 is now known as mjturek12:42
*** hemna has quit IRC12:46
*** sapd1 has quit IRC12:46
*** ociuhandu has quit IRC12:46
*** hemna has joined #openstack-nova12:47
*** stephenfin has quit IRC12:49
*** stephenfin has joined #openstack-nova13:08
*** luksky has quit IRC13:11
*** luksky has joined #openstack-nova13:11
*** luksky has quit IRC13:12
*** luksky has joined #openstack-nova13:12
*** Habib-A has joined #openstack-nova13:14
Habib-AHello13:14
Habib-Ai have a running production environment of Openstack with the following architecture13:14
Habib-Ait was installed manually13:14
Habib-A1 controller node13:14
Habib-A1 network node13:14
Habib-A3 physical compute nodes13:14
Habib-A1 cinder node13:14
Habib-Ais it possible to add HA to environment ? is it easy ? will cause data loss ?13:14
yoctozeptoHabib-A: it's probably not the right channel to ask this question; it really depends on the deployment method you used (kolla/osa/tripleo/charms/helm...)13:15
yoctozeptoI assume you mean controller HA13:15
Habib-Ai did say i installed it manually13:16
Habib-Aim sorry if its the wrong place13:16
yoctozeptoit is usually quite straightforward and does not cause any data loss13:16
yoctozeptoah, I missed that, sorry13:16
yoctozeptoI am into misreading today it seems13:16
Habib-Ais it fine even though its currently in production and the network node is not on the controller13:17
Habib-Ano problem my friend thank you very much for your help13:17
yoctozeptoyes, the network node can be separate13:17
yoctozeptoyou can have different numbers of each of these13:17
sean-k-mooneyHabib-A: i would proably make the netwo contol and cinder nodes all contollers13:18
yoctozepto^ yeah, depending on your needs you might find yourselves happy setting these as your HA backups13:18
sean-k-mooneyyou would do that by installing rabbit mq on all 3 nodes and making it a cluster behind a ha proxy instance13:19
sean-k-mooneythen do the same with the db add 2 more instance and use galera with haproxy in front13:19
sean-k-mooneythen deploy  2 more instance of each of the contoelr/network/cinder services13:19
sean-k-mooneyagain using haproxy to loadbalance between them13:20
sean-k-mooneythat would be the most simple ha setup13:20
Habib-Athats very cool many thanks13:20
sean-k-mooneyHabib-A: even if you are doing a manual install i would recommend looking at kolla-ansible as a reference13:20
sean-k-mooneyor consider moving to it :)13:20
Habib-Ahmmm i get you13:21
Habib-Acan i pm for more details >13:21
Habib-A?13:21
*** Luzi has quit IRC13:21
sean-k-mooneyi have never actully adopted an existing manual install into one of the deployment tools13:22
sean-k-mooneyand i generally use them to provide ha so i dont think i can help direclty13:22
sean-k-mooneyHabib-A: https://docs.openstack.org/ha-guide/ should help13:22
Habib-A my question is how can i move the cinder and network to the controller13:23
Habib-Awithout losing data13:23
sean-k-mooneyif you were interested in using kolla-ansible #openstack-kolla would proably be the best place to ask if you consier one of the other tool there channles too would be where you would ask13:23
sean-k-mooneyHabib-A: basically by doing a db backup and restore to the second and thrid db instance then froming a galare cluster making your current db the primary/leader13:24
Habib-Ai want to move to kolla but is it possible to migrate my currnet environment to the new environment13:25
sean-k-mooneyHabib-A: yes i know people have done it in the past im not sure if its documented13:25
sean-k-mooneyHabib-A: kolla support external dbs13:25
Habib-Ai understand the DB part but for the network node and cinder13:25
sean-k-mooneywell the network nodes dont store data locally13:25
Habib-Aok that makes sense13:26
Habib-Awhat about cinder13:26
sean-k-mooneyso really you just install more api/l3/dhcp/l2 agents on the cinder and contler node13:26
sean-k-mooneycinder depneds on the backend13:26
Habib-Alvm213:26
sean-k-mooneyare you using ceph? lvm?13:26
sean-k-mooneyok so that is harder13:26
Habib-Aand its in a VM (vmware )13:26
sean-k-mooney you can make the api/db ha13:27
Habib-Ai get you13:27
sean-k-mooneythe lvm dirver itslef does not support ha13:27
Habib-Agot it13:27
Habib-Athank you very much sean :D13:27
sean-k-mooneyas in you cant have 2 cinder volume agent managne the same set of lvm volumes13:27
sean-k-mooneyyou can however have many seperate backend13:27
sean-k-mooneyHabib-A: the path forwad for cinder would be to deploy ceph or another ha backend13:28
sean-k-mooneythen do a volume migration form the lvm backend to the new one13:28
sean-k-mooneyif you were using a backend that supported ha volumes like ceph you would have jsut installed addtion cinder volume agents and pointed them all at the same ceph cluster with the same config13:29
Habib-Ahmmm makes sense13:30
sean-k-mooneyHabib-A: the simplest way to move an exsiting deployment to kolla ansibel is to use external dbs by the way13:34
sean-k-mooneyhttps://github.com/openstack/kolla-ansible/blob/master/doc/source/reference/databases/external-mariadb-guide.rst13:34
openstackgerritLee Yarwood proposed openstack/nova stable/train: api: Reject volume attach requests when an active bdm exists  https://review.opendev.org/c/openstack/nova/+/78584813:35
*** waleedm has joined #openstack-nova13:38
*** ociuhandu has joined #openstack-nova13:38
*** amodi has joined #openstack-nova14:00
*** waleedm has quit IRC14:06
*** waleedm has joined #openstack-nova14:07
*** k_mouza has quit IRC14:24
*** k_mouza has joined #openstack-nova14:38
dansmithsean-k-mooney: gibi: stephenfin: Can you think of any reason this is still defaulted to false? https://github.com/openstack/nova/blob/master/nova/conf/scheduler.py#L15214:45
sean-k-mooneyi think i have a patch to change that14:45
sean-k-mooneyi was looking at removign the filter14:45
dansmithIIRC we introduced it as false because we had to wait a cycle for mirroring of the aggregates, but at this point I would think we'd want that on14:45
dansmithsean-k-mooney: right okay, I was surprised to find it still =False and thought the filter would have been deprecated by now14:45
dansmithowalsh: ^14:45
sean-k-mooneyhttps://review.opendev.org/c/openstack/nova/+/74560514:46
sean-k-mooneyi need to repopose the blueprint but i can do that14:46
dansmithcool14:48
*** ociuhandu has quit IRC14:52
*** ociuhandu has joined #openstack-nova14:54
*** dpawlik has quit IRC14:58
*** ociuhandu has quit IRC14:58
*** macz_ has joined #openstack-nova15:01
*** efried has joined #openstack-nova15:15
*** ociuhandu has joined #openstack-nova15:24
*** ociuhandu has quit IRC15:32
*** ociuhandu has joined #openstack-nova15:35
*** ociuhandu has quit IRC15:40
*** mlavalle has joined #openstack-nova15:42
*** k_mouza has quit IRC15:43
*** ociuhandu has joined #openstack-nova15:44
*** xinranwang has quit IRC15:46
*** derekh has quit IRC15:56
*** jamesdenton has quit IRC16:03
*** ociuhandu has quit IRC16:07
*** ociuhandu has joined #openstack-nova16:07
*** vishalmanchanda has quit IRC16:10
*** lucasagomes has quit IRC16:12
*** ociuhandu has quit IRC16:12
*** ociuhandu has joined #openstack-nova16:13
*** macz_ has quit IRC16:19
*** k_mouza has joined #openstack-nova16:21
openstackgerritMerged openstack/python-novaclient master: Add unit tests for client logger  https://review.opendev.org/c/openstack/python-novaclient/+/78574116:21
*** tesseract has quit IRC16:23
*** ociuhandu_ has joined #openstack-nova16:26
*** k_mouza has quit IRC16:27
*** k_mouza has joined #openstack-nova16:27
*** ociuhandu has quit IRC16:29
*** jamesdenton has joined #openstack-nova16:30
*** ociuhandu_ has quit IRC16:31
openstackgerritLee Yarwood proposed openstack/nova stable/train: api: Reject volume attach requests when an active bdm exists  https://review.opendev.org/c/openstack/nova/+/78584816:39
*** waleedm has quit IRC16:46
*** hamalq has joined #openstack-nova16:47
*** hamalq has quit IRC16:47
*** hamalq has joined #openstack-nova16:48
elodlyarwood: out of curiosity, do you want this patch because of the fix of bug #1879878 or because other patch? >>> https://review.opendev.org/c/openstack/nova/+/751364/516:50
openstackbug 1879878 in OpenStack Compute (nova) train "VM become Error after confirming resize with Error info CPUUnpinningInvalid on source node " [Undecided,In progress] https://launchpad.net/bugs/1879878 - Assigned to Stephen Finucane (stephenfinucane)16:50
elod(and sorry for pinging you only now :X)16:50
lyarwoodelod: I *think* we are carrying this downstream so both bug #1879878 and making any future backports easier upstream in stable/train16:51
openstackbug 1879878 in OpenStack Compute (nova) train "VM become Error after confirming resize with Error info CPUUnpinningInvalid on source node " [Undecided,In progress] https://launchpad.net/bugs/1879878 - Assigned to Stephen Finucane (stephenfinucane)16:51
elodlyarwood: I see. As I was not fond of backporting that amount of test refactors... I waited to approve these test refactors until the bugfix part is reviewed by more eyes, to reduce the risk of introducing regressions16:55
lyarwoodelod: I understand but we are actually reducing the amount of code churn in backports by pulling back something like this16:58
elodyes, for train, but for stein, we have to do the churn anyway o:)16:58
elodbut I understand that you only support train16:58
lyarwoodwell we support Queens still but only really target Train at the moment16:59
lyarwoodbut yeah eventually we will have to avoid the backport and just do the hard work but I'd prefer to defer that as long as possible16:59
*** rpittau is now known as rpittau|afk16:59
lyarwoodwe already suffer on stable/train without _create_server etc in the func tests16:59
lyarwoodbut anyway, I don't see the harm when it's this self contained tbh17:00
*** k_mouza_ has joined #openstack-nova17:00
lyarwoodlet me hit the tip of the series17:00
lyarwoodand we can go from there17:00
elodso as a standalone patch I wouldn't really merge it. but I understand the need... :S17:01
elodalso train-em transition is closer and closer. if it's not a risk, then it's okay to be part of the last release, otherwise I would consider waiting until the transition is done17:02
*** k_mouza has quit IRC17:02
*** dtantsur is now known as dtantsur|afk17:06
*** links has quit IRC17:11
*** macz_ has joined #openstack-nova17:15
*** macz_ has quit IRC17:17
*** macz_ has joined #openstack-nova17:17
*** dklyle has joined #openstack-nova17:24
*** MrClayPole has quit IRC17:35
*** MrClayPole has joined #openstack-nova17:35
*** ralonsoh has quit IRC17:52
*** efried has quit IRC18:00
sean-k-mooneyelod: so add them after its EM or just not at all18:01
*** efried has joined #openstack-nova18:01
sean-k-mooneyelod: context is we need to support train for 3 more years so it would be nice to have them as we backport regression tests18:01
sean-k-mooney* downstream18:02
openstackgerritRodrigo Barbieri proposed openstack/nova master: Error anti-affinity violation on migrations  https://review.opendev.org/c/openstack/nova/+/78416618:03
elodsean-k-mooney: yes, I understand that and accept the reason. I just wanted to avoid regression (especially now when train will trainsition to extended maintenance). that's why I wanted more eyes on the bugfix part.18:14
sean-k-mooneysure i was just asking if you were ok with it going in after it was EM or if you wanted to not backport the _create_server and start_compute functions18:15
elodwell, as I said, I'm not fond of backporting such test refactors, but if stable cores are fine with it then we can merge it18:18
elodand we can even merge it now, if it is considered not risky18:18
elod(as I am maybe overcautious in this big patch series... :X)18:19
elod(this one: https://review.opendev.org/q/topic:%22bug%252F1879878%22 )18:21
sean-k-mooneywell https://review.opendev.org/c/openstack/nova/+/751364 is kind of indepent of that18:22
sean-k-mooneyits a funciontal test change only so it cant break production code18:23
*** ociuhandu has joined #openstack-nova18:23
elodyes it is, but it is in the middle of a big patch series o:)18:24
sean-k-mooneytrue altouh its number 3 in the seriese and the other patches before it are like wise test only changes18:25
sean-k-mooneyi do understand the caution18:26
sean-k-mooneywhich is why i did not backport them myself the last time to avoid having to disucss the merrits or them and focus just on the minium code change for the bug i was fixing18:27
*** ociuhandu has quit IRC18:29
*** k_mouza_ has quit IRC18:31
elodwell, as I understood we don't even backport standalone test refactoring patches without any reason, so if they are not bug related18:39
*** andrewbonney has quit IRC18:47
sean-k-mooneydansmith: gibi i dont know if i had a bp or bug before for the az filter but i filed https://blueprints.launchpad.net/nova/+spec/remove-deprecated-filters now to track it in either case for xena18:50
sean-k-mooneyi dont know if ill get to it before the ptg but ill see if i can  get around to it tomorrow18:51
dansmithokay I'd probably argue we don't need a spec for this.. assume you're just planning a bp?18:51
sean-k-mooneyoh ya planning a specless blueprint18:51
dansmithack18:51
sean-k-mooneyi just want something to track it in launchpad and a bug felt weird18:52
*** ociuhandu has joined #openstack-nova19:16
*** k_mouza has joined #openstack-nova19:32
*** k_mouza has quit IRC19:37
*** whoami-rajat has quit IRC19:55
*** ociuhandu has quit IRC20:11
*** slaweq has quit IRC20:38
*** slaweq has joined #openstack-nova20:41
*** macz_ has quit IRC20:47
*** hamalq has quit IRC21:11
*** hamalq has joined #openstack-nova21:11
*** macz_ has joined #openstack-nova21:44
*** slaweq has quit IRC21:53
*** luksky has quit IRC22:49
*** artom has quit IRC22:54
*** artom has joined #openstack-nova22:54
*** rcernin has joined #openstack-nova23:04
*** tosky has quit IRC23:24
*** artom has quit IRC23:29
yonglihereview request: Repropose smartnic support spec:  https://review.opendev.org/c/openstack/nova-specs/+/78363223:53
*** macz_ has quit IRC23:58
« Sunday, 2021-04-11 Index Tuesday, 2021-04-13 »

Generated by irclog2html.py 2.17.2 by Marius Gedminas - find it at https://mg.pov.lt/irclog2html/!