Monday, 2019-11-18

*** dsneddon_ has quit IRC00:04
*** bfournie has joined #openstack-ironic00:05
openstackgerritJulia Kreger proposed openstack/ironic-specs master: Ussuri project priorities  https://review.opendev.org/69470400:31
openstackgerritJulia Kreger proposed openstack/ironic-python-agent master: WIP: RAID 5/6  https://review.opendev.org/69470500:50
kaifengmorning TheJulia o/00:52
kaifengthe raid 5/6 seems working in this way in theory, but in my memory raid 1 needs even number disks00:53
kaifengbasically we just need to add extra validations, and pass correct arguments to mdadm00:54
TheJuliao/00:58
TheJuliakaifeng: you can do a 3 disk raid 1, you just have 3 copies instead of 100:58
TheJuliaI think00:58
TheJuliathere _was_ a point where mdadm wouldn't do it, but I remember doing it on a 1U dell server years ago that had three drive bays00:59
kaifengTheJulia: in that case, that should work, i can check whether mdadm complains with raid 1 on 3 disks :)00:59
TheJuliakaifeng: That would be awesome01:00
kaifengTheJulia: I am kind of sign in the Experiment goal :)01:03
TheJuliasign on? :)01:03
TheJuliain also means on in Latin... as well as in.01:03
kaifengthey may be a trivial difference to non-english users, yeah, i think i mean sign up (or sign on)01:04
TheJulia\o/01:05
kaifengTheJulia: raid 1 with odd disks works \o/ i must be overthinking01:22
TheJulia\o/01:22
*** ociuhandu has joined #openstack-ironic01:31
*** chenhaw has joined #openstack-ironic01:35
*** ociuhandu has quit IRC01:41
*** trungnv has joined #openstack-ironic01:42
*** trungnv has quit IRC01:49
*** goldyfruit_ has quit IRC01:53
*** dsneddon_ has joined #openstack-ironic02:00
*** dsneddon_ has quit IRC02:05
*** ociuhandu has joined #openstack-ironic02:26
*** ociuhandu has quit IRC02:30
*** ericin has joined #openstack-ironic02:48
*** ociuhandu has joined #openstack-ironic03:00
*** ericin has quit IRC03:02
*** ericin has joined #openstack-ironic03:02
*** ericleiin has joined #openstack-ironic03:06
*** ociuhandu has quit IRC03:07
*** ericin has quit IRC03:09
*** ericlei_ has joined #openstack-ironic03:10
*** ericleiin has quit IRC03:13
*** ericleiin has joined #openstack-ironic03:19
*** ericlei_ has quit IRC03:22
*** mkrai has joined #openstack-ironic03:34
*** ericlei_ has joined #openstack-ironic03:34
*** ericleiin has quit IRC03:37
*** mkrai_ has joined #openstack-ironic03:40
*** mkrai has quit IRC03:41
*** ericlei_ has quit IRC03:52
*** ericlei_ has joined #openstack-ironic03:52
cpWhat is the status of RedFish API support in Ironic ?03:57
*** dsneddon_ has joined #openstack-ironic04:01
*** dsneddon_ has quit IRC04:05
rpiosocp: At least a couple of ironic drivers, which ironic calls hardware types, support Redfish. I’m aware of the generic Redfish and iDRAC drivers, named ‘redfish’ and ‘idrac’, respectively. idrac is for use with Dell EMC bare metal systems with an integrated Dell Remote Access Controller.04:20
rpiosocp: Other vendor-specific drivers may also support Redfish.04:21
rpiosocp: Please see https://docs.openstack.org/ironic/latest/admin/drivers.html and its linked driver-specific pages for details.04:23
*** ociuhandu has joined #openstack-ironic04:27
*** ociuhandu has quit IRC04:29
*** ociuhandu has joined #openstack-ironic04:30
*** ociuhandu has quit IRC04:36
cprpioso: thanks a lot for these information04:40
cprpioso: ironic can also discover new hardware automatically/periodically ?04:41
cprunning ipmi-discover or ssdp ?04:42
rpiosocp: I don’t have direct experience with that. Others in this channel should be able to answer those questions.04:43
cpwhat is the best way to handle discovery and enrolment of new hardware in a remote location ( remote to the main OpenStack )04:43
cplike an edge DC04:43
cpare there any solutions for this ?04:43
*** ociuhandu has joined #openstack-ironic04:53
*** ociuhandu has quit IRC04:58
*** ociuhandu has joined #openstack-ironic04:58
*** mkrai_ has quit IRC04:59
*** ociuhandu has quit IRC05:03
*** mkrai has joined #openstack-ironic05:11
*** mkrai has quit IRC05:14
*** mkrai has joined #openstack-ironic05:14
*** mkrai_ has joined #openstack-ironic05:22
*** mkrai has quit IRC05:24
kaifengcp: i think we currently don't have the support for periodically discoverying05:25
kaifengautomatical discovery is possible, as long as the bare metal boots from the ramdisk with proper callback address to the ironic-inspector.05:28
cpbare metal boots from nothing05:29
cpthe only thing they have is a BMC connected to the IPMI/Redfish management network05:29
cpthis is the only assumption that is possible05:30
*** ociuhandu has joined #openstack-ironic05:30
cpkaifeng: so when the machine are plugged in the BMC comes alive and is "accessible" because it has gotten an IP from the DHCP server05:31
kaifengwell, i feel that's unlikely for ironic to do a discovery without manual intervention.05:31
cpkaifeng: then you can discover it with SSDP05:31
cpor IPMI-discover05:31
cpkaifeng: why not ?05:31
cpkaifeng: ah you mean , this feature is not available in ironic at the moment05:32
cpso probably I will have to write some module myself to do the automatic enrolment05:32
kaifengfor ironic, we need basic oob information to manage the node05:32
cpassumption is that OOB network exists05:33
cp(IPMI/Redfish/Swordfish)05:33
cpkaifeng: I have to study what is needed by Ironic for enrollment05:34
*** ociuhandu has quit IRC05:35
cpI also need to create a keystone user probably05:35
kaifengi don't have experience on the ipam on bmc network though, if you can get the ip address/credentials by external means, then autoenrollment is possible too, but i think it's not in the ironic part.05:35
cpkaifeng: ah you mean this has to be in NOVA ?05:36
kaifengcp: no, nova has no knowledge on this05:36
kaifengfor enrollment of a node, essentially ironic needs ipmi address/credential, and a driver desired to use.05:38
cpThis is the part I am struggling with , the interaction between what I write and OpenStack05:38
cpok so I can use the ironic API05:39
kaifengcp: definitely, and it's the only way to enroll a node :)05:39
cpI am wondering if there is a better way to do this for a remote site that needs to be controlled by OpenStack than to enroll it in the core Openstack (centralized)05:40
cpkaifeng: thanks for the clarification05:40
kaifengcp: yw, let's think this reversively, how ironic can manage a node without enough information?05:40
cpI can provide it all the information it needs05:41
cpWhy would I need to thing reversively ?05:41
cpkaifeng: maybe you have thought about something , I did not think about05:42
cps/,//05:42
cpCan you explain what you mean in more details ?05:43
kaifengif i understand it right, you want to manage a remote site but don't want to enroll the node to openstack.05:44
cpI do want to enroll it in openstack05:44
kaifengok, maybe I am misreading something.05:45
cpI am just wondering is there is a better way to do than enrolling it in the centralized OpenStack05:45
cpmaybe there is a way to decentralize OpenStack in whole of part while still being able to manage it all from one central location05:46
cpfor example I want to deploy images locally from the remote site for the machines that are in that site05:46
cpI do not want have to go fetch images over the wire05:47
cpto the central site05:47
cpIn short the less traffic I have over my link to the remote site the better it is05:48
cpI want the remote site to be somewhat autonomous for provisioning operations that require lots of bandwidth05:48
cpProbably I can point at a glance server cluster that is in the remote site ?05:49
kaifengcp: ironic can be used without other services, you can set up http servers for accessing images on the remote site05:49
cpkaifeng: I see05:50
kaifengthat will avoid images been transfered from cloud center05:50
cpwhat about the Horizon UI ?05:51
cpwill it be controlable from it still ?05:51
cp(in the case ironic is used alone)05:52
kaifengi guess no, but not so sure, horizon talks to api directly, which i think ironic needs to be registered to keystone.05:53
cpI wonder how people manage remote sites with Openstack05:54
kaifengi feel people in starlingx may have experience on this05:56
*** dsneddon_ has joined #openstack-ironic06:02
*** dsneddon_ has quit IRC06:06
*** jtomasek has joined #openstack-ironic06:26
openstackgerritKaifeng Wang proposed openstack/ironic master: Remove old online migration codes  https://review.opendev.org/69471806:41
rpiosocp: You may find Distributed Compute Nodes (DCN) interesting.06:42
*** yolanda has quit IRC06:45
cpstarlingx seems to not have the experience06:52
cprpioso: I will look at that up06:52
mkrai_good morning ironic o/06:58
mkrai_Hi kaifeng cp06:58
kaifenghey mkrai_ o/06:58
mkrai_rpioso, Hi o/06:59
rpiosoGood morning, mkrai_07:00
mkrai_I am seeing error in ipa that it couldn't reach the controller node as the node is not assigned any IP. ANy idea what could be the possible issue?07:00
mkrai_kaifeng, rpioso any pointer to debug?07:01
*** ociuhandu has joined #openstack-ironic07:02
openstackgerritMerged openstack/ironic master: Block ability update callback_url  https://review.opendev.org/69077507:02
kaifengmkrai_: you mean the ramdisk didn't get ip address?07:04
*** rcernin has quit IRC07:04
mkrai_yes07:04
mkrai_on loopback interface exist07:04
mkrai_*only07:04
kaifengmkrai: i guess you still running devstack, have you checked the xml of the vm?07:05
mkrai_kaifeng, yes I am running devstack but booting an actual baremetal node07:06
*** ociuhandu has quit IRC07:06
kaifengmkrai: well, i think that might mean the ramdisk doesn't have device drivers could drive the nic hardware.07:07
mkrai_kaifeng, Ok got it. Thanks! Let me check that07:09
rpiosomkrai_: Ensure the bare metal NIC port is configured to PXE boot.07:12
rpiosoThe one on the provisioning network.07:13
* rpioso goes to sleep07:13
*** rpioso is now known as rpioso|afk07:13
mkrai_rpioso, Yes it is :) Thanks07:13
arne_wiebalckGood morning, ironic!07:26
arne_wiebalckTheJulia: Not crazy at all :) The RAID levels are artificially limited, mostly b/c I did not have a way to test RAID5/6 on real machines and b/c of the error handling (as RAID5/6 are a little more demanding in terms of disks). Apart from this, it should be totally fine to extend the frozen set.07:28
*** rpittau|afk is now known as rpittau07:28
rpittaugood morning ironic! o/07:29
arne_wiebalckGood morning rpittau o/07:30
kaifenggood morning arne_wiebalck rpittau o/07:33
arne_wiebalckkaifeng: RAID-1 works with 3 disks as well (we ran like this for some time)07:33
arne_wiebalckhey kaifeng o/07:33
rpittauhey arne_wiebalck kaifeng :)07:34
kaifengarne_wiebalck: your right, tested earlier today, my knowledge on raid maybe too old :)07:34
kaifengarne_wiebalck: btw can i have hardware raid 1 on odd disks?07:35
arne_wiebalckkaifeng: I think 3 (or more) disks is simply unusual, but it works fine :)07:35
rpittauyou'll have 3 copies instead of 2 :)07:35
arne_wiebalckkaifeng: I guess that may depend on the controller. The algorithm allows this.07:36
arne_wiebalckrpittau: right ... super safe! :-D07:36
kaifengyep, that make sense to me, but i just have a memory that it must be even numbered, maybe it's a limit on controller implementation.07:36
arne_wiebalckkaifeng: that's totally possible07:37
rpittaukaifeng: it's probably a limit in the controller, there's no such limit AFAIR07:37
kaifengGet it, thanks :)07:38
rpittaurequires breakfast07:42
*** igordc has joined #openstack-ironic07:58
*** igordc has quit IRC08:02
*** dsneddon_ has joined #openstack-ironic08:03
*** mkrai_ has quit IRC08:06
*** dsneddon_ has quit IRC08:08
*** tesseract has joined #openstack-ironic08:15
*** ociuhandu has joined #openstack-ironic08:22
*** ociuhandu has quit IRC08:29
openstackgerritKaifeng Wang proposed openstack/ironic master: Drop [agent]heartbeat_timeout  https://review.opendev.org/69472608:30
*** mkrai has joined #openstack-ironic08:42
*** Lucas_Gray has joined #openstack-ironic08:42
cpmkrai: hi :)08:43
dtantsurmorning ironic08:44
kaifenghi dtantsur o/08:45
rpittauhey dtantsur :)08:45
*** lucasagomes has joined #openstack-ironic08:45
*** mkrai has quit IRC08:50
*** mkrai has joined #openstack-ironic08:51
gudrutishi all08:52
*** stendulker has joined #openstack-ironic09:00
*** ociuhandu has joined #openstack-ironic09:04
kaifenggudrutis o/09:05
openstackgerritMerged openstack/networking-baremetal master: Add genconfig env to tox  https://review.opendev.org/69449209:05
*** dougsz has joined #openstack-ironic09:09
openstackgerritKaifeng Wang proposed openstack/ironic master: Drop [agent]heartbeat_timeout  https://review.opendev.org/69472609:10
*** alexmcleod has joined #openstack-ironic09:20
*** yolanda has joined #openstack-ironic09:22
szivianiGood morning, ironic :)09:31
*** stendulker has quit IRC09:31
*** iurygregory has joined #openstack-ironic09:41
iurygregorygood morning Ironic!09:41
rpittauhey sziviani iurygregory :)09:41
iurygregoryrpittau, o/09:41
*** mkrai has quit IRC09:42
*** bobmel has joined #openstack-ironic09:45
*** mkrai has joined #openstack-ironic09:54
*** mkrai has quit IRC09:56
*** ociuhandu has quit IRC09:57
*** mkrai has joined #openstack-ironic09:57
*** ociuhandu has joined #openstack-ironic09:58
*** ociuhandu has quit IRC10:03
*** dsneddon_ has joined #openstack-ironic10:04
*** lennyb has quit IRC10:06
*** mkrai has quit IRC10:07
*** dsneddon_ has quit IRC10:08
*** pcaruana has joined #openstack-ironic10:10
iurygregoryI love the random failures on ironic-standalone =D10:34
*** khansa has joined #openstack-ironic10:45
*** rcernin has joined #openstack-ironic10:52
openstackgerritMerged openstack/ironic master: Allow node owners to administer nodes  https://review.opendev.org/68955110:53
openstackgerritMerged openstack/ironic-python-agent master: Bump up ipa-ip-lookup-attempts to 6  https://review.opendev.org/69364010:53
*** chenhaw has quit IRC10:59
*** ociuhandu has joined #openstack-ironic11:02
*** mkrai has joined #openstack-ironic11:05
*** dougsz has quit IRC11:08
dtantsuriurygregory: who doesn't? :)11:14
*** rachit7 has joined #openstack-ironic11:23
openstackgerrithuwenhui proposed openstack/ironic-tempest-plugin master: fix bug for unit test  https://review.opendev.org/69451111:36
*** rachit7 has quit IRC11:39
*** mkrai has quit IRC11:44
*** dougsz has joined #openstack-ironic11:47
*** mkrai has joined #openstack-ironic11:49
*** bobmel has quit IRC11:52
*** mkrai has quit IRC11:54
*** mkrai_ has joined #openstack-ironic11:54
*** dsneddon_ has joined #openstack-ironic12:05
*** dsneddon_ has quit IRC12:09
*** ociuhandu has quit IRC12:13
iurygregorydtantsur, morning o/ and it's even better with the rainy weather in Brno =)12:17
*** mkrai_ has quit IRC12:19
*** rcernin has quit IRC12:31
dtantsurit rains cats and dogs here too :)12:34
etingofalmost rain forest over here12:37
*** ociuhandu has joined #openstack-ironic12:43
*** sziviani has quit IRC12:46
*** sziviani has joined #openstack-ironic12:49
dtantsurbeware of anacondas!12:51
iurygregorydtantsur, WUT?!12:51
iurygregoryO.o12:51
*** dsneddon_ has joined #openstack-ironic12:53
*** ericlei_ has quit IRC13:00
*** Lucas_Gray has quit IRC13:02
* etingof can deal with Python13:03
*** ociuhandu has quit IRC13:04
*** lennyb has joined #openstack-ironic13:09
*** rh-jelabarre has joined #openstack-ironic13:09
*** dtantsur is now known as dtantsur|bbl13:21
openstackgerritMerged openstack/ironic master: Stop supporting incompatible heartbeat interfaces  https://review.opendev.org/69447313:30
*** rloo has joined #openstack-ironic13:34
etingofwhy do we have VMs in CI jobs having different amount of RAM?13:38
rpittauetingof: I can only thnk about different scopes for the VMs13:41
rpittauetingof: which job(s) ?13:42
etingofrpittau, a handful of them, this setting makes them different -- https://github.com/openstack/ironic/blob/master/zuul.d/ironic-jobs.yaml#L4713:43
openstackgerritkhansa proposed openstack/sushy master: Add OEM extension example script  https://review.opendev.org/68983513:44
rpittauetingof: yep, that's the base job, so depending on the job more RAM is required for the VM13:45
iurygregorydepends on the scope of the test etc, tempest tests that will run and other things13:46
rpittauit's seem consistent, uefi jobs have VMs with 512 and centos7 has 3072 (!!!)13:47
*** whoami-rajat has joined #openstack-ironic13:47
etingofrpittau, iurygregory so why uefi partition and bios wholedisk VMs need more RAM?13:47
rpittauetingof: I don't see bios jobs with more RAM13:48
iurygregoryetingof, i do remember we had to upgrade to ram for some jobs because of KP in the VMS13:48
*** khansa has quit IRC13:48
rpittauwell, more like VMs suddenly stopping working and blanking out :D13:49
etingofKP?13:49
rpittauKernel Panic13:50
etingofah, panic13:50
etingofrpittau, right, for bios it's 384 as for base. it's just that the setting is duplicated13:50
iurygregoryXD13:50
etingofrpittau, however for uefi it's 512 -- https://github.com/openstack/ironic/blob/master/zuul.d/ironic-jobs.yaml#L52413:50
rpittauetingof: I'm pretty sure 384 is not enough for UEFI13:51
rpittauit was tested and we had to increase it13:51
rpittauif there are duplicate variables, we should definitely remove them13:51
etingofrpittau, why uefi needs more ram?13:51
etingofthe boot loader itself is so large?13:51
*** hwoarang has quit IRC13:52
etingofmy vmedia jobs keep failing mysteriously. the last think on the console is 'initrd...ok'13:52
rpittauetingof: I don't remember the details, I think it's a combination of factors13:52
etingofso I am thinking to pump up some ram13:52
rpittauetingof: do it! :)13:53
etingofI am pessimistic because with the same amount of RAM the same job works in devstack (c)13:53
openstackgerritIlya Etingof proposed openstack/ironic master: Add a CI job to legacy boot over Redfish virtual media  https://review.opendev.org/69301713:55
rpittauetingof: ok, I get it, so on your devstack env works, but on upstream CI13:55
etingofrpittau, not in upstream CI, right13:55
etingofVM behave weirdly - boot loader boots, but not loads13:56
*** Goneri has joined #openstack-ironic13:57
*** hwoarang has joined #openstack-ironic13:58
rpittauetingof: I see you increased the RAM in the latest set, let's see how that goes13:59
etingofyeah14:00
*** lennyb has quit IRC14:14
*** Lucas_Gray has joined #openstack-ironic14:23
*** Lucas_Gray has quit IRC14:25
*** Lucas_Gray has joined #openstack-ironic14:28
*** beekneemech is now known as bnemec14:29
*** rachit7 has joined #openstack-ironic14:31
*** tzumainn has joined #openstack-ironic14:32
*** goldyfruit has joined #openstack-ironic14:40
*** cdearborn has joined #openstack-ironic14:44
*** dougsz has quit IRC14:45
*** goldyfruit_ has joined #openstack-ironic14:51
*** stendulker has joined #openstack-ironic14:52
*** goldyfruit has quit IRC14:53
*** kaifeng_ has joined #openstack-ironic14:54
*** rachit7 has quit IRC14:56
TheJuliaGood morning everyone14:57
etingofo/ TheJulia14:58
arne_wiebalckHey TheJulia o/14:58
TheJuliawait... anacondas in Brno?!?14:58
* TheJulia sure there is someone that worked on anaconda there... 14:58
rpittauhey TheJulia :)15:00
TheJuliaIs it time for a meeting?15:00
rpittauyes!15:00
* etingof is lost in timezones15:00
TheJulia#startmeeting ironic15:00
openstackMeeting started Mon Nov 18 15:00:44 2019 UTC and is due to finish in 60 minutes.  The chair is TheJulia. Information about MeetBot at http://wiki.debian.org/MeetBot.15:00
openstackUseful Commands: #action #agreed #help #info #idea #link #topic #startvote.15:00
*** openstack changes topic to " (Meeting topic: ironic)"15:00
openstackThe meeting name has been set to 'ironic'15:00
TheJuliaetingof: me toooooooo15:00
etingofo/15:00
kaifeng_morning TheJulia (again) o/15:00
rpittauo/15:00
TheJuliaetingof: and I think I need to be on a boat in like an hour15:01
cdearborno/15:01
arne_wiebalcko/15:01
mgoddard\o15:01
*** tesseract has quit IRC15:01
bdoddo/15:01
TheJuliaGood morning everyone!15:01
*** tesseract has joined #openstack-ironic15:01
kaifeng_o/15:01
TheJuliaOur agenda this week has a few items, and those can be found on the wiki.15:01
*** dougsz has joined #openstack-ironic15:01
* arne_wiebalck will need to leave half-way through the meeting15:01
*** dtantsur|bbl is now known as dtantsur15:01
TheJulia#link https://wiki.openstack.org/wiki/Meetings/Ironic#Agenda_for_next_meeting15:01
dtantsuro/15:01
bfourniebfournie: o/15:01
TheJulia#topic Announcements / Reminder15:01
*** openstack changes topic to "Announcements / Reminder (Meeting topic: ironic)"15:01
stendulkero/15:02
TheJuliaI have to items to announce15:02
rpioso|afko/15:02
TheJuliaThe first is that I sent out a summary of the ironic related sessions and the PTG to the mailing list.15:02
TheJulia#link http://lists.openstack.org/pipermail/openstack-discuss/2019-November/010789.html15:02
*** rpioso|afk is now known as rpioso15:02
dtantsurTheJulia++15:02
TheJuliaThe second, I'm now in San Diego for Kubecon15:02
arne_wiebalckThanks a lot TheJulia !15:03
rpittauTheJulia: the summary is really cool :)15:03
TheJuliaSo it may take me a little while to reply this week.15:03
TheJuliaThanks everyone!15:03
mgoddard+1, thanks15:03
TheJuliadtantsur: did we have any action items from last week?15:03
TheJulialooks like there were none15:03
rlooo/15:04
jroll\o15:04
rpittauTheJulia: the naval yard in San Diego is awesome, if you have time :)15:04
dtantsurTheJulia: not that I can think of15:04
TheJuliaNormally I would say we proceed to looking at subteam status reports, but I've proposed a new priority list for this cycle15:04
TheJulia#link https://review.opendev.org/#/c/694704/15:05
patchbotpatch 694704 - ironic-specs - Ussuri project priorities - 2 patch sets15:05
TheJuliaAny objection to going directly to priorities for the week?15:06
dtantsurnone from me15:06
* TheJulia suspects she could find crickets15:07
TheJuliaOkay then!15:07
TheJulia#topic Priorities for the week15:07
*** openstack changes topic to "Priorities for the week (Meeting topic: ironic)"15:07
TheJulia#link https://etherpad.openstack.org/p/IronicWhiteBoard15:08
TheJuliaLine 17115:08
TheJuliamgoddard: you seem to be going back and forth on https://review.opendev.org/#/c/672252/, is a -1 required to signal revision required?15:09
patchbotpatch 672252 - ironic-specs - Add support for node reconfigure - 5 patch sets15:09
* iurygregory is late o/15:09
TheJuliao/ iurygregory15:09
mgoddardTheJulia: yeah, I'll add one15:09
*** stendulker_ has joined #openstack-ironic15:10
TheJuliaWhile I'm deleting/updating entires there, does anyone have patches to propose?15:10
*** stendulker has quit IRC15:11
dtantsurmine are there15:11
TheJuliaarne_wiebalck: I'm going to move xXraphXx's IPA patches down to the bottom of the list since they haven't been rebased/updated yet :(15:12
iurygregoryI'm working on the patches to drop the py27 jobs going to push during the week but no hurry =)15:12
dtantsurtzumainn: are you planning on the remaining ownership patches this week?15:12
dtantsurI could take the owned allocations, it's a fair chunk of work15:12
tzumainndtantsur, hi! I'm planning on working on the one for ports, followed by allocations15:12
arne_wiebalckTheJulia: sure ... it's still on my list :)15:12
etingofmine parches are there as well15:12
dtantsurtzumainn: ah, lemme know if you want to do allocations yourself15:12
tzumainndtantsur, I wouldn't mind taking a shot!15:12
tzumainnunless you are somehow supremely bored15:12
dtantsurwonderful! I can help with reviews/guidance.15:12
tzumainnokay, sounds good!15:13
dtantsurI'm never bored :)15:13
tzumainnhahaha, I figured :)15:13
dtantsur(I am sometimes bored, but from procrastination, not from lack of tasks)15:13
TheJuliaawesome https://review.opendev.org/#/c/689551/ merged15:14
patchbotpatch 689551 - ironic - Allow node owners to administer nodes (MERGED) - 7 patch sets15:14
TheJuliaI feel like there should be a release soon!15:14
mgoddardthat can be arranged15:14
mgoddardwhich project(s)?15:14
TheJulialets wait, I need to work something out with the stable team first :\15:14
mgoddardok15:14
TheJuliaI ran out of time last week :(15:14
TheJuliaWell if there is nothing else to add to the list, I think it looks fairly good for this week and we can proceed to Discussion15:15
dtantsurtzumainn: JFYI another thing I'm pondering (well, not only me, it's been a recurring topic) is a single-endpoint deployment API (rather than a combination of node update + VIF + node provision state).15:15
dtantsurI can try putting together a spec for that this week15:16
dtantsurTheJulia: ^^15:16
tzumainndtantsur, ah, that'd definitely be of interest!15:16
TheJuliaI'd personally prefer to see us focus on some performance issues in our API, but... if there is interest15:17
dtantsurTheJulia: which exactly? I think I may have some time (wow) this cycle.15:17
dtantsurbut I also think that the ownership work is semi-useful without this new deployment API15:17
rpiosoTheJulia: Re: Baremetal SIG, line 297, I believe you've found the OSF POCs. Right?15:17
dtantsurbecause we don't want non-privileged users to have node update rights15:18
*** dsneddon_ has quit IRC15:18
TheJuliadtantsur: this is true, if we can somehow improve the performance to get a list of nodes and then get nodes, it would be a good thing... HOW exactly, needs to be determined, but I've had a few people grumble about how long it takes to list out machines in large deployments :\15:19
dtantsurI'm not sure there is any magic to squeeze there other than telling people to use 'fields' extensively15:19
TheJuliaMaybe we just need to be mindful15:20
*** TxGirlGeek has joined #openstack-ironic15:20
TheJuliarpioso: danny@openstack.org and wes@openstack.org15:20
dtantsurbut, well, I can put it on my TODO list. maybe I can figure out something using x000 of fake nodes15:20
rpiosoTheJulia: ty15:20
TheJuliadtantsur: yeah, one item in talking with nova at the ptg was "hey, if we can load 1000 nodes into ironic, and then see how long it takes for those to get into resource tracker, we can begin to somewhat measure this15:20
dtantsurtrivial to do if somebody tells me how to reliably define "gets into resource tracker"15:21
TheJuliaWow my lag is getting really bad from irccloud15:21
TheJulia#topic Discussion15:21
*** openstack changes topic to "Discussion (Meeting topic: ironic)"15:21
TheJuliadtantsur: well, placement actually15:22
arne_wiebalckTheJulia: dtantsur: I think the nova team decided that it would be much more sensible to use conductor groups.15:22
TheJuliaTwo topics for this week15:22
arne_wiebalckTo address scaling issues.15:22
TheJuliaarne_wiebalck: I've not followed that thread, and that does not surprise me15:22
arne_wiebalckTheJulia: We will give that a try in our deployment.15:22
TheJuliaI can likely take an action item to try and describe what that looks like, at least the way it was intended to be used15:22
dtantsuryeah, I've seen it. also our effort will be limited when it comes to fixing nova (as opposed to our API)15:22
dtantsurTheJulia: I sense another reference architecture! (which we'll be too busy to finish, sigh)15:23
arne_wiebalckThe RT code seems to be somewhat complicated.15:23
jroll"somewhat" :)15:23
*** dsneddon_ has joined #openstack-ironic15:23
dtantsur:D15:23
arne_wiebalckAnd there are worries to break things for the non-ironic use case (which is arguably the main use case).15:24
arne_wiebalckjroll: ;)15:24
jrolldtantsur: re "gets into resource tracker", timing this method is probably the best starting point: https://opendev.org/openstack/nova/src/branch/master/nova/compute/manager.py#L918515:24
TheJuliadtantsur: well, I would agree with a reference architecture but the overall code review process applyign to docs kind of kills that from working because eveyrone is differently opinionated15:24
TheJuliaanyway, discussion items15:24
jrollthat's the periodic task15:24
jrolloh sorry, I thought this was open discussion :)15:24
dtantsuryeah, let's get back to it later :)15:24
TheJulia#info There is an open question of if we would like to have a mid-cycle. We've been offered space in Central Europe for two days. Feb 25-26 maybe?15:25
dtantsurEurope \o/15:25
TheJuliaI don't expect everyone to have a "yes" answer, but I think it would be a good thing to organize since the next official gathering with be in Vancouver in June15:26
TheJuliaPlease consult with your management, look at prices, and maybe we can revisit the question next week?15:26
*** ociuhandu has joined #openstack-ironic15:26
* iurygregory CERN? =P15:26
dtantsurI'm tentatively yes (assuming budget approval)15:26
arne_wiebalckiurygregory: yes15:26
dtantsurTheJulia: do you have a rough idea which Central Europe? prices can be VERY different15:26
dtantsurCERN would be awesome and very hard budget-wise at the same time15:27
TheJuliaGVA15:27
etingofmust be a geographical center of Europe...15:27
jrolloh boy, I need to find a way to justify this (I won't be able to, sigh)15:27
* TheJulia is sensing ?maybe?15:28
* dtantsur would probably go to GVA even on his own budget15:28
iurygregoryjroll, we are all in the same boat =)15:28
*** ociuhandu has quit IRC15:28
*** dsneddon_ has quit IRC15:28
TheJuliaI did look, it is only an 8 hour train ride from CDG15:28
* iurygregory would do the same thing XD15:28
dtantsuriurygregory: well, he also needs an intercontinental flight15:28
rpittauI would definitely go no matter what :D15:28
jrolliurygregory: not all, I'm sure arne can justify it :D15:28
rloowhat about virtual mid-cycle?15:28
dtantsurvirtual mid-cycles are cheap, we can have both15:29
iurygregory^ ++15:29
arne_wiebalckWe're happy to host a mid-cycle here at CERN. Most important thing for now would be to settle on dates.15:29
arne_wiebalckAs I'd need to book the room.15:29
arne_wiebalck(and organize tours ;-)15:30
*** rachit7 has joined #openstack-ironic15:30
rpittauend of february seems ok, it's in the middle between now and the next summit15:30
TheJulia(to understand how our software is used ;))15:30
iurygregorybut we have the dates, we just don't know about how many can make it15:30
TheJuliaAnyway, we should kind of move on to the next topic15:30
rpiosoAny FOSS conferences around that time?15:30
dtantsurFOSDEM in early Feb15:31
TheJuliaSomewhere somebody has to have a central conference calendar15:31
TheJuliaAnyway15:31
TheJuliaNext discussion topic15:31
rpiosodtantsur: Thx15:32
TheJuliaIt has been raised to our attention that the nova virt driver in ironic lacks documentation.15:32
TheJulia#link https://bugs.launchpad.net/nova/+bug/185244615:32
openstackLaunchpad bug 1852446 in OpenStack Compute (nova) "Hypervisors in nova - no subpage details for ironic" [Undecided,New]15:32
TheJuliaWhile surely some might just link into ironic's documentation, is anyone interested in maybe taking a stab at adding some documentation to nova?15:33
jrollmaybe we should link it to this page until we do write some? https://docs.openstack.org/ironic/latest/install/configure-compute.html15:33
TheJuliaSeems a good first step15:34
iurygregoryI'm interested, just would like to know if this has to be done this week (next week I have time to start working on this)15:34
TheJuliaWe don't want to duplicate everything, the audience is different there too15:34
jroll++15:34
iurygregoryalso it depends on how much details we want there of course15:34
TheJuliaiurygregory: does not need to be done this week15:34
iurygregoryand maybe I will bother people with questions XD15:35
TheJuliaiurygregory: you going to take a shot at putting something together?15:35
TheJuliaSounds good :)15:35
iurygregoryTheJulia, sure =)15:35
TheJuliaMoving on!15:36
TheJulia#topic Baremetal SIG15:37
*** openstack changes topic to "Baremetal SIG (Meeting topic: ironic)"15:37
TheJuliaFirst item, the white paper!15:37
TheJuliaarne_wiebalck: The bare metal logo program participants submitted case studies topic came up in a discusison with foundation folks last week.15:38
arne_wiebalckTheJulia: Can we use them as case study input?15:38
TheJuliaarne_wiebalck: They are going to iterate through participants and likely send introduction emails for the SIG that you should be getting with a reminder that they should submit their content to the whitepaper for case study input15:39
arne_wiebalckTheJulia: Nice!15:39
TheJuliaSo at least a forward step there!15:39
TheJuliaRemember, anyone can help work on the whitepaper!15:39
arne_wiebalckWe'll need some "glue text" with the larger picture.15:40
TheJuliaarne_wiebalck: I guess that kind of covers the first two items :)15:40
rpiosoTheJulia: \o/15:40
arne_wiebalckTheJulia: It does, thanks!15:40
TheJuliaarne_wiebalck: I'm already happy to help with glue text15:40
arne_wiebalckTheJulia: Awesome, thanks!15:40
dtantsurare there any tasks requiring average English skills? :)15:40
arne_wiebalckdtantsur: always! :)15:41
arne_wiebalckFor the ops meetup, one of the things we said at the PTG was to show some presence at these events ... anyone planning to attend?15:41
TheJuliaOne other item for the SIG that Arne and I both feel would be good, is that at least one ironic team contributor coul dmake it to the Operators Meetup in London on January 7th-8th15:41
dtantsurLondon.. likely nope15:41
dtantsurnone of those needs a visa can make it15:42
dtantsur* those who need15:42
rpittauit's a couple hours train from Paris :P15:42
etingofforbidden lands15:42
arne_wiebalckOne issue I had with these meetings in the past was that the agenda was built very late.15:42
* dtantsur appoints rpittau15:42
rpittaulol15:42
arne_wiebalckSo, it is somewhat hard to know what to expect (and justify in time).15:42
mgoddardI might be able to attend15:42
dtantsurfor the record, I enjoyed the ops meetup in Berlin. a nice opportunity to hear what smaller operators are thinking15:43
arne_wiebalckdtantsur: I was in Milan. While it is nice to meet other ops, it is good when project cores are around.15:43
rpittauI might be also able to attend :)15:43
TheJuliaAlso, they tend to be separate industry verticals. The NYC meetup I attended a few months back were mostly financials and their only ask was "help them figure out a migration path from..."15:43
TheJuliacobbler I think15:44
rpittauoh gosh15:44
iurygregoryouch =X15:44
* TheJulia senses rpittau is backing away from the computer now15:44
TheJulia:)15:44
rpittaukind of :D15:44
dtantsurmigration from cobbler came up in Berlin too15:45
TheJuliaNo RFE's listed to review, so lets go to Open Discussion15:45
TheJulia#topic Open Discussion15:45
*** openstack changes topic to "Open Discussion (Meeting topic: ironic)"15:45
* arne_wiebalck has to leave, sorry o/15:45
TheJuliadtantsur: interesting... perhaps a blog post or something is... required15:45
TheJuliao/ arne_wiebalck15:45
rpittaubye arne_wiebalck o/15:45
rpiosoarne_wiebalck: \o15:45
rpittauofficial cobbler-ironic migration guide?15:46
* TheJulia backs away from the computer a little15:46
TheJuliamaybe?15:46
TheJuliaWe have so many things that need work though :(15:46
kaifeng_maybe simply because they are not in the help wanted :P15:47
*** stendulker_ has quit IRC15:48
rachit7Hi TheJulia dtantsur rpittau please add this patch in your review list:  https://review.opendev.org/#/c/671038/15:49
patchbotpatch 671038 - ironic - DRAC: Drives conversion from raid to jbod - 21 patch sets15:49
*** dsneddon_ has joined #openstack-ironic15:49
dtantsurit's already on vendor priorities15:49
TheJuliakaifeng_: it is a possibility, or enough of the major problems have been solved15:50
TheJuliakaifeng_: also help wanted is a whole nightmare at the openstack level15:50
kaifeng_i am feeling we have many information scattered around etherpads but lacking a full list15:51
rpittauI was wondering if we should convert the etherpad with the CI failures to a more persistent doc15:52
TheJuliafor ironic, it feels more like we don't ask for help early enough, or that sometimes we ask and people volunteer, and then those people get pulled downstream for weeks or months :(15:53
rpittauI mean the issues resolved etherpad15:53
TheJuliaWhich is not their fault :(15:53
TheJuliarpittau: perhaps?15:53
TheJuliaSome of those failures have occassionally appeared for operators, so it would be visibility I guess15:54
TheJuliaand google indexable15:54
rpittaumaybe a wiki section ?15:54
TheJuliawell15:54
TheJulia$favoritesearchengineindexable15:54
TheJuliaLots of people in the grass roots linux community seem to be trying to figure out ways to de-google15:54
TheJuliaWiki might work15:55
*** dsneddon_ has quit IRC15:55
kaifeng_is there any requirement for the white paper cases?15:56
TheJuliakaifeng_: no requirement other than a willingness to share perspective15:58
TheJuliaAnyway, our meeting time is almost over15:59
TheJuliaThanks everyone15:59
dtantsurthank you!15:59
openstackgerritDmitry Tantsur proposed openstack/ironic-python-agent-builder master: Generate checksums for DIB images  https://review.opendev.org/69480015:59
iurygregoryty15:59
*** dsneddon_ has joined #openstack-ironic16:00
rpittauthanks!16:00
cdearborntx16:00
*** hamzy has quit IRC16:01
TheJulia#endmeeting16:01
*** openstack changes topic to "Bare Metal Provisioning | Status: http://bit.ly/ironic-whiteboard | Docs: http://docs.openstack.org/ironic/ | Bugs: https://storyboard.openstack.org/#!/project_group/75 | Contributors are generally present between 6 AM and 12 AM UTC, If we do not answer, please feel free to pose questions to openstack-discuss mailing list."16:01
openstackMeeting ended Mon Nov 18 16:01:13 2019 UTC.  Information about MeetBot at http://wiki.debian.org/MeetBot . (v 0.1.4)16:01
openstackMinutes:        http://eavesdrop.openstack.org/meetings/ironic/2019/ironic.2019-11-18-15.00.html16:01
openstackMinutes (text): http://eavesdrop.openstack.org/meetings/ironic/2019/ironic.2019-11-18-15.00.txt16:01
openstackLog:            http://eavesdrop.openstack.org/meetings/ironic/2019/ironic.2019-11-18-15.00.log.html16:01
mgoddardFor anyone running nova with multiple computes, here is an interesting bug: https://bugs.launchpad.net/nova/+bug/185300916:01
openstackLaunchpad bug 1853009 in OpenStack Compute (nova) "Ironic node rebalance race can lead to missing compute nodes in DB" [Undecided,In progress] - Assigned to Mark Goddard (mgoddard)16:01
mgoddardjroll, arne_wiebalck, dtantsur, TheJulia ^16:01
jrollooo fun16:02
*** gyee has joined #openstack-ironic16:02
jrollmgoddard: we were seeing similar (on ocata), but there was a patch we backported that seemed to help16:03
jrolllemme find it16:03
mgoddardhave a fix in progress16:03
jrollmgoddard: this one https://review.opendev.org/#/c/579922/16:03
patchbotpatch 579922 - nova - Delete orphan compute nodes before updating resources (MERGED) - 3 patch sets16:03
jrollwe haven't seen this happen since we applied that16:03
mgoddardjroll: got it :)16:04
mgoddard(as in, we are already using that patch)16:04
iurygregoryhey everyone o/ I know the meeting is over but I would like to get a discussion about https://storyboard.openstack.org/#!/story/200684716:05
jrollhuh, interesting16:05
iurygregoryThis only happens when we want to use UEFI + secure boot16:06
etingofrpittau, no luck with 512M of RAM as well - kernel dies after printing out 'S'16:06
rpittauetingof: fantastic...16:06
etingofwhere 'S' stands for Starting perhaps16:07
iurygregorybasically there is a bug with grub2-install and we should use efibootmgr if secure boot is enable, I was thinking in verify this with mockutil, but suggestions are welcome =)16:08
*** kaifeng_ has quit IRC16:08
mgoddardjroll: https://review.opendev.org/#/c/694802/16:10
patchbotpatch 694802 - nova - [ironic] Fix node rebalance race issues - 1 patch set16:10
dtantsuriurygregory: is there a way to fix grub2-install? I cannot believe we're the only one with this problem.16:12
TheJuliaiurygregory: your going to have to mock the calls regardless, so I'm not sure other than defining the new path and tests around the command sequence we _should_ be using if the bits are already prsent16:12
jrollmgoddard: thanks, will look16:12
TheJuliadtantsur: no, by design it rebuilds the files which causes signing to be dropped16:12
dtantsurI'm surprised it doesn't come up in quick googling..16:13
TheJuliaAlso, the other issue that actually was found with that is that there is a legit bug in grub2-install where nvram updating misbehaves in some fun cases. I believe grub folks are working on that16:13
TheJuliafun cases being where no default record exists previously or something like that, and efibootmgr apparently gracefully deals with that or something16:14
TheJuliasomething something *magic*16:14
*** ociuhandu has joined #openstack-ironic16:14
iurygregoryTheJulia, my idea would be verify if secure boot is enable using mokutil ( I saw this https://www.kvaser.com/developer-blog/secure-boot-linux-systems/) and if the condition is valid start we would call efibootmgr to do the job =)16:15
TheJuliaiurygregory: wouldn't it be better to focus on if the bits are already on the disk image?16:15
TheJuliaif bits there, then update efi nvram16:16
TheJuliaif not, lets do the install16:16
dtantsurwhy don't we use just efibootmgr on UEFI systems?16:16
TheJuliadtantsur: because if partition image, we have to put the bits in place16:16
TheJulia:\16:16
dtantsurTheJulia: but it won't work for secure boot, will it?16:17
dtantsurdoes it mean that we cannot support partition images with secure boot?16:17
TheJuliadtantsur: in that case, true, not really16:17
iurygregoryTheJulia, so before even installing grub2 we would verify the bits to see if everything is ok?16:17
TheJulianot necessarilly16:17
TheJuliadepends on a few different factors16:17
TheJuliaiurygregory: That is the direction I started taking with mbr booting16:18
*** rachit7 has quit IRC16:19
*** ociuhandu has quit IRC16:19
TheJuliaanyway, I need to get going, bbl16:20
*** sziviani has quit IRC16:22
*** whoami-rajat has quit IRC16:27
*** ociuhandu has joined #openstack-ironic16:30
iurygregoryanyone else with some thoughts?16:33
dtantsurwell, at least for whole disk images we should stop doing grub2-install altogether16:37
dtantsurit seems that partition images just won't work with secure boot though16:38
iurygregoryack16:40
rpiosoetingof, TheJulia: I received an email asking me to upload our Summit preso so it can be made publicly available. Has that already been done? If not, I would be glad to handle it.16:48
etingofrpioso, not me16:48
*** hamzy has joined #openstack-ironic16:51
openstackgerritMerged openstack/ironic master: Switch legacy jobs to Py3  https://review.opendev.org/69406016:52
*** aedc has joined #openstack-ironic16:59
*** lucasagomes has quit IRC17:03
iurygregory\o/ Thanks CI17:07
rpittauwas about time :)17:07
iurygregoryyeah17:07
iurygregorygoing to double check if we missed other legacy jobs and start pushing drop-py27 \o/17:07
*** Lucas_Gray has quit IRC17:09
*** iurygregory has quit IRC17:15
*** ociuhandu has quit IRC17:17
rpittaugood night! o/17:18
*** rpittau is now known as rpittau|afk17:18
*** hamzy has quit IRC17:21
*** hamzy has joined #openstack-ironic17:22
*** ociuhandu has joined #openstack-ironic17:28
openstackgerritIlya Etingof proposed openstack/ironic master: Add a CI job to UEFI boot over Redfish virtual media  https://review.opendev.org/69386717:29
*** dsneddon_ has quit IRC17:32
*** dsneddon_ has joined #openstack-ironic17:34
*** dsneddon_ has quit IRC17:39
*** ociuhandu has quit IRC17:40
*** bobmel has joined #openstack-ironic17:51
*** dtantsur is now known as dtantsur|afk18:03
dtantsur|afko/18:03
*** dsneddon_ has joined #openstack-ironic18:11
*** hjensas has quit IRC18:36
*** igordc has joined #openstack-ironic18:46
*** tesseract has quit IRC18:52
*** alexmcleod has quit IRC18:54
*** hjensas has joined #openstack-ironic19:11
*** aedc has quit IRC19:29
*** dsneddon_ has quit IRC19:35
*** dsneddon_ has joined #openstack-ironic19:40
openstackgerritJulia Kreger proposed openstack/ironic-specs master: Ussuri project priorities  https://review.opendev.org/69470419:42
*** dsneddon_ has quit IRC19:48
*** dsneddon_ has joined #openstack-ironic19:50
*** dsneddon_ has quit IRC19:55
*** dsneddon_ has joined #openstack-ironic20:06
*** dsneddon_ has quit IRC20:11
*** bobmel has quit IRC20:17
*** dsneddon_ has joined #openstack-ironic20:50
*** dsneddon_ has quit IRC20:58
*** goldyfruit_ has quit IRC21:02
*** dsneddon_ has joined #openstack-ironic21:04
*** dougsz has quit IRC21:05
*** dsneddon_ has quit IRC21:08
*** goldyfruit has joined #openstack-ironic21:17
*** ociuhandu has joined #openstack-ironic21:38
*** dsneddon_ has joined #openstack-ironic21:40
*** ociuhandu has quit IRC21:43
*** hwoarang has quit IRC22:24
*** pcaruana has quit IRC22:26
*** hwoarang has joined #openstack-ironic22:28
*** zaneb has joined #openstack-ironic22:39
*** rcernin has joined #openstack-ironic22:46
*** zaneb has quit IRC23:25
*** ociuhandu has joined #openstack-ironic23:31
*** ociuhandu has quit IRC23:36
*** goldyfruit has quit IRC23:36
*** dsneddon_ has quit IRC23:59

Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!