| *** cdearborn has quit IRC | 00:12 | |
| *** ijw_ has joined #openstack-ironic | 00:20 | |
| *** gyee has quit IRC | 00:22 | |
| *** ijw has quit IRC | 00:23 | |
| *** ijw_ has quit IRC | 00:24 | |
| openstackgerrit | Merged openstack/ironic master: Adding file uri support for ipa image location https://review.opendev.org/678729 | 01:14 |
|---|---|---|
| *** jcoufal has joined #openstack-ironic | 01:28 | |
| *** jcoufal has quit IRC | 01:31 | |
| *** ricolin has joined #openstack-ironic | 01:48 | |
| *** ricolin has quit IRC | 02:04 | |
| *** ricolin has joined #openstack-ironic | 02:05 | |
| *** dsneddon has quit IRC | 02:16 | |
| *** ricolin has quit IRC | 02:32 | |
| *** ricolin has joined #openstack-ironic | 02:32 | |
| *** openstackgerrit has quit IRC | 02:37 | |
| *** dsneddon has joined #openstack-ironic | 02:45 | |
| *** dsneddon has quit IRC | 03:09 | |
| *** rcernin has quit IRC | 03:13 | |
| *** rloo has quit IRC | 03:16 | |
| *** ricolin has quit IRC | 03:21 | |
| *** yaawang has quit IRC | 03:26 | |
| *** yaawang has joined #openstack-ironic | 03:27 | |
| *** dsneddon has joined #openstack-ironic | 03:44 | |
| *** dsneddon has quit IRC | 03:49 | |
| *** gkadam has joined #openstack-ironic | 03:50 | |
| *** gkadam has quit IRC | 03:50 | |
| *** rcernin has joined #openstack-ironic | 04:05 | |
| *** dsneddon has joined #openstack-ironic | 04:24 | |
| *** mkrai has joined #openstack-ironic | 04:25 | |
| *** dsneddon has quit IRC | 04:30 | |
| *** rachit7 has joined #openstack-ironic | 04:49 | |
| *** dsneddon has joined #openstack-ironic | 05:04 | |
| *** openstackgerrit has joined #openstack-ironic | 05:09 | |
| openstackgerrit | Shivanand Tendulker proposed openstack/ironic master: Add iLO RAID deploy steps https://review.opendev.org/674271 | 05:09 |
| *** dsneddon has quit IRC | 05:09 | |
| *** pradip_k has joined #openstack-ironic | 05:15 | |
| openstackgerrit | Shivanand Tendulker proposed openstack/ironic master: Add deploy steps for iLO BIOS interface https://review.opendev.org/674270 | 05:16 |
| openstackgerrit | Shivanand Tendulker proposed openstack/ironic master: WIP: Add iDRAC RAID deploy steps https://review.opendev.org/641731 | 05:17 |
| pradip_k | Hi Shivanand, I have one query regarding https://review.opendev.org/#/c/674021/8 patch, I am updating patch by adding try/except clause for "clear_job_queue" clean step, Do I need to add try/except clause for ''reset_idrac'' and "known_good_state" clean steps in this patch as well?? | 05:23 |
| patchbot | patch 674021 - ironic - DRAC : clear_job_queue clean step to fix pending b... - 8 patch sets | 05:23 |
| *** dsneddon has joined #openstack-ironic | 05:35 | |
| *** dsneddon has quit IRC | 05:40 | |
| *** devx has quit IRC | 05:48 | |
| *** devx has joined #openstack-ironic | 05:50 | |
| *** rcernin_ has joined #openstack-ironic | 05:52 | |
| *** pradip_k has quit IRC | 05:54 | |
| *** pradip_k has joined #openstack-ironic | 05:54 | |
| *** pradip_k has quit IRC | 05:54 | |
| *** pradip_k has joined #openstack-ironic | 05:55 | |
| *** rcernin has quit IRC | 05:55 | |
| rachit7 | Good Morning Ironicers..!!! | 05:58 |
| *** pradip_k has quit IRC | 06:02 | |
| *** pradip_k has joined #openstack-ironic | 06:02 | |
| *** pradip_k has quit IRC | 06:08 | |
| *** pradip_k has joined #openstack-ironic | 06:08 | |
| *** dsneddon has joined #openstack-ironic | 06:10 | |
| *** pradip_k has quit IRC | 06:12 | |
| *** dsneddon has quit IRC | 06:16 | |
| *** dsneddon has joined #openstack-ironic | 06:17 | |
| *** dsneddon has quit IRC | 06:22 | |
| openstackgerrit | Chason Chan proposed openstack/virtualbmc master: Tiny fix for documentation RST convention https://review.opendev.org/679187 | 06:25 |
| *** pradip_k has joined #openstack-ironic | 06:27 | |
| arne_wiebalck | Good morning, ironic! | 06:30 |
| *** ricolin has joined #openstack-ironic | 06:38 | |
| *** dougsz has joined #openstack-ironic | 06:48 | |
| kaifeng | gm arne_wiebalck o/ | 06:57 |
| *** dsneddon has joined #openstack-ironic | 06:57 | |
| arne_wiebalck | hey kaifeng o/ | 06:58 |
| *** dsneddon has quit IRC | 07:03 | |
| *** pradip_k has quit IRC | 07:11 | |
| *** rachit7 has quit IRC | 07:12 | |
| *** dsneddon has joined #openstack-ironic | 07:33 | |
| *** lucasagomes has joined #openstack-ironic | 07:37 | |
| *** pradip_k has joined #openstack-ironic | 07:39 | |
| *** dougsz has quit IRC | 07:40 | |
| *** rcernin_ has quit IRC | 07:40 | |
| *** trident has quit IRC | 07:40 | |
| *** dsneddon has quit IRC | 07:41 | |
| *** dougsz has joined #openstack-ironic | 07:43 | |
| *** trident has joined #openstack-ironic | 07:49 | |
| openstackgerrit | Ankit Kumar proposed openstack/ironic master: Add iso less virtual media deploy https://review.opendev.org/677996 | 07:53 |
| *** dougsz has quit IRC | 07:53 | |
| *** priteau has joined #openstack-ironic | 07:54 | |
| *** rcernin_ has joined #openstack-ironic | 07:57 | |
| *** dsneddon has joined #openstack-ironic | 08:10 | |
| *** e0ne has joined #openstack-ironic | 08:11 | |
| *** dsneddon has quit IRC | 08:15 | |
| *** pradip_k has quit IRC | 08:17 | |
| *** e0ne has quit IRC | 08:20 | |
| *** alexmcleod has joined #openstack-ironic | 08:33 | |
| *** e0ne has joined #openstack-ironic | 08:34 | |
| *** dsneddon has joined #openstack-ironic | 08:44 | |
| *** dsneddon has quit IRC | 08:49 | |
| *** rachit7 has joined #openstack-ironic | 08:54 | |
| openstackgerrit | Dmitry Tantsur proposed openstack/ironic-python-agent-builder master: DIB: install IPA from source checked out by Zuul https://review.opendev.org/679086 | 08:58 |
| *** e0ne has quit IRC | 08:59 | |
| *** pradip_k has joined #openstack-ironic | 08:59 | |
| openstackgerrit | Pradip Kadam proposed openstack/ironic master: DRAC : clear_job_queue clean step to fix pending bios config jobs https://review.opendev.org/674021 | 09:08 |
| *** dsneddon has joined #openstack-ironic | 09:20 | |
| *** dsneddon has quit IRC | 09:24 | |
| *** e0ne has joined #openstack-ironic | 09:27 | |
| *** e0ne has quit IRC | 09:27 | |
| arne_wiebalck | Is there currently a way to limit parallel deployments? When creating hundreds of instances, the controllers seem to run out of memory, apparently when running qemu-img. I went from 3 to 5 controllers, hoping to reduce the parallelism, and will go to bigger nodes next, but I was just wondering if it would make sense to look into making it work with small nodes (by self-limiting ironic). | 09:37 |
| *** dsneddon has joined #openstack-ironic | 09:53 | |
| *** dsneddon has quit IRC | 09:58 | |
| openstackgerrit | M V P Nitesh proposed openstack/ironic-python-agent master: Creates and Reorders the Boot Entry https://review.opendev.org/669832 | 10:06 |
| *** derekh has joined #openstack-ironic | 10:20 | |
| *** e0ne has joined #openstack-ironic | 10:21 | |
| *** dtantsur|afk is now known as dtantsur | 10:24 | |
| dtantsur | morning ironic | 10:24 |
| *** e0ne has quit IRC | 10:27 | |
| *** dsneddon has joined #openstack-ironic | 10:29 | |
| *** jawad_axd has joined #openstack-ironic | 10:29 | |
| *** andrein has joined #openstack-ironic | 10:34 | |
| *** dsneddon has quit IRC | 10:34 | |
| openstackgerrit | M V P Nitesh proposed openstack/ironic master: Creates boot entry on bare metal in UEFI boot mode for wholedisk images https://review.opendev.org/670266 | 10:37 |
| *** mkrai has quit IRC | 10:53 | |
| *** mkrai has joined #openstack-ironic | 10:54 | |
| *** mkrai has quit IRC | 10:57 | |
| *** mkrai has joined #openstack-ironic | 10:57 | |
| *** e0ne has joined #openstack-ironic | 11:00 | |
| *** dsneddon has joined #openstack-ironic | 11:00 | |
| *** mkrai has quit IRC | 11:05 | |
| *** dsneddon has quit IRC | 11:05 | |
| *** pradip_k has quit IRC | 11:05 | |
| openstackgerrit | Dmitry Tantsur proposed openstack/ironic-python-agent master: DNM testing the new CI jobs https://review.opendev.org/679106 | 11:07 |
| *** tesseract has joined #openstack-ironic | 11:15 | |
| dtantsur | TheJulia: https://review.opendev.org/#/c/679233/ | 11:15 |
| patchbot | patch 679233 - networking-generic-switch-tempest-plugin - Abandon this repository - 1 patch set | 11:15 |
| *** Lucas_Gray has joined #openstack-ironic | 11:19 | |
| *** e0ne has quit IRC | 11:20 | |
| *** e0ne has joined #openstack-ironic | 11:23 | |
| *** e0ne has quit IRC | 11:32 | |
| *** rh-jelabarre has joined #openstack-ironic | 11:34 | |
| *** rachit7 has quit IRC | 11:35 | |
| *** dsneddon has joined #openstack-ironic | 11:37 | |
| TheJulia | dtantsur: thanks | 11:40 |
| *** e0ne has joined #openstack-ironic | 11:40 | |
| dtantsur | np.. | 11:40 |
| * TheJulia makes French press style coffee this morning | 11:40 | |
| dtantsur | TheJulia: FYI it seems that this https://github.com/openstack/ironic/blob/master/zuul.d/ironic-jobs.yaml#L385 doesn't actually work | 11:41 |
| dtantsur | and I'm struggling to find a way to override OS_TEST_TIMEOUT for new-style jobs (coreos ones were old style) | 11:41 |
| *** dsneddon has quit IRC | 11:41 | |
| TheJulia | Is there a general env cars parameter? | 11:43 |
| TheJulia | My phone things vars should be cara | 11:43 |
| TheJulia | Bah | 11:43 |
| *** Lucas_Gray has quit IRC | 11:45 | |
| *** jcoufal has joined #openstack-ironic | 11:45 | |
| kaifeng | goodnight TheJulia, dtantsur o/ | 11:57 |
| dtantsur | g'night | 11:58 |
| *** andrein has quit IRC | 11:58 | |
| *** andrein has joined #openstack-ironic | 11:59 | |
| TheJulia | Good night | 12:00 |
| * TheJulia has another cup of coffee to wake up | 12:00 | |
| *** e0ne has quit IRC | 12:11 | |
| *** dsneddon has joined #openstack-ironic | 12:14 | |
| openstackgerrit | Ilya Etingof proposed openstack/sushy-tools master: Do not reuse sqlite connection https://review.opendev.org/678797 | 12:17 |
| *** dsneddon has quit IRC | 12:19 | |
| openstackgerrit | Dmitry Tantsur proposed openstack/ironic master: WIP ipv6 jobs https://review.opendev.org/613169 | 12:21 |
| openstackgerrit | Dmitry Tantsur proposed openstack/ironic-python-agent-builder master: Import ansible version of the image building job https://review.opendev.org/678884 | 12:23 |
| *** derekh has quit IRC | 12:23 | |
| *** dsneddon has joined #openstack-ironic | 12:26 | |
| TheJulia | arne_wiebalck: still hunting a raid issue | 12:28 |
| *** jcoufal has quit IRC | 12:28 | |
| TheJulia | in your config, do you have [agent]manage_agent_boot set to true or false? | 12:28 |
| *** jcoufal has joined #openstack-ironic | 12:30 | |
| *** dsneddon has quit IRC | 12:31 | |
| *** jcoufal has quit IRC | 12:33 | |
| *** rloo has joined #openstack-ironic | 12:35 | |
| arne_wiebalck | TheJulia: we do not override the default (so we use true) | 12:36 |
| TheJulia | hmm | 12:36 |
| TheJulia | and your absolutely sure no kernel and ramdisk are present on the images your deploying? | 12:36 |
| TheJulia | what is your default deploy interface? | 12:37 |
| arne_wiebalck | TheJulia: iscsi | 12:38 |
| TheJulia | hmmm | 12:38 |
| * arne_wiebalck notices that the hmmm gets bigger and bigger | 12:38 | |
| TheJulia | hmmmm | 12:38 |
| TheJulia | indeed | 12:38 |
| arne_wiebalck | TheJulia: what is the actual failure | 12:38 |
| TheJulia | Well, I went to change the default logic for raid, and hit: https://b1dbb34e22ddc2f6b152-1659b40e5c03e7f989419d6178d67ae8.ssl.cf1.rackcdn.com/679132/1/check/ironic-standalone/97c5492/testr_results.html.gz | 12:39 |
| arne_wiebalck | TheJulia: this is during deployment (not cleaning), I guess? | 12:39 |
| TheJulia | yes, during deployment | 12:39 |
| TheJulia | same issue as changing the default_boot_option to local in the conductor | 12:40 |
| * TheJulia wonders if instances are being fired up with a netboot flavor or something | 12:40 | |
| * TheJulia is running out of m's for her hmmm | 12:40 | |
| *** Lucas_Gray has joined #openstack-ironic | 12:40 | |
| *** andrein has quit IRC | 12:42 | |
| arne_wiebalck | I'm not really familiar with tempest, but is there a way to see what is going on on the node (as the log only shows a timeout for not reaching active, from what I see)? | 12:43 |
| arne_wiebalck | Like: would a broken grub result in such an erorr? | 12:44 |
| TheJulia | arne_wiebalck: really only digging through the conductor log. I guess I'm really wondering if your deployment is always setting up a boot loader or not. | 12:44 |
| arne_wiebalck | TheJulia: It should not. | 12:44 |
| arne_wiebalck | TheJulia: Only if RAID is configured. | 12:44 |
| TheJulia | well, so the tests were previously getting 'netboot' for prepare_instance_to_boot's, so it was skipping installing a boot loader | 12:44 |
| TheJulia | only IF raid is configured, so it could be that the tempest tests are just broken then | 12:45 |
| arne_wiebalck | TheJulia: the target raid config has the "controller=software" flag to trigger the boot loader installation | 12:45 |
| arne_wiebalck | TheJulia: our images are full disk images and have hence the bootloader built-in | 12:46 |
| TheJulia | okay | 12:47 |
| arne_wiebalck | TheJulia: So, we always have a bootloader ofc, but only when we deploy on a software RAID ironic installs it. | 12:47 |
| TheJulia | so... this is an issue of our CI images don't actually have grub-install | 12:47 |
| * TheJulia tableflips | 12:47 | |
| TheJulia | and for $reasons, it just happens to still boot | 12:48 |
| TheJulia | dtantsur: ^^^ any thoughts on this. the tl;dr that hte tempest tests work in CI because the boot_option default ends up being netboot, they don't ever actually setup to local boot really as a result | 12:49 |
| TheJulia | dtantsur: so when bootloader install occurs, there is no grub-install | 12:49 |
| openstackgerrit | Julia Kreger proposed openstack/ironic master: Fix get_boot_option logic https://review.opendev.org/679132 | 12:50 |
| TheJulia | which is why ^^^ fails too | 12:51 |
| TheJulia | oh, and I guess the partition selection code is actually incompatible with the cirros whole disk images | 12:52 |
| TheJulia | because there is not an actual partition | 12:52 |
| TheJulia | So crazy question, what if we just looked at the header of the image to see if it was bootable? | 12:52 |
| TheJulia | maybe the answer is to kill cirros with fire | 12:57 |
| *** dsneddon has joined #openstack-ironic | 12:57 | |
| *** derekh has joined #openstack-ironic | 12:58 | |
| arne_wiebalck | the partition selection code to find a place to install the bootloader? | 12:59 |
| dtantsur | TheJulia: I'll read the scrollback later today, okay? Having some important errands right now (may be healing nerves mid vodka later on) | 12:59 |
| dtantsur | s/mid/with/ (a very interesting German-English mix) | 12:59 |
| TheJulia | dtantsur: understood, thanks :) | 12:59 |
| * TheJulia groked it | 13:00 | |
| TheJulia | so the raw cirros disk image does actually pave a partition... | 13:00 |
| arne_wiebalck | hm | 13:00 |
| * arne_wiebalck checks which image Rally is using to test our ironic deployment | 13:01 | |
| TheJulia | arne_wiebalck: I suspect you guys are good | 13:02 |
| *** dsneddon has quit IRC | 13:02 | |
| arne_wiebalck | TheJulia: Rally does not test s/w RAID anyway | 13:02 |
| arne_wiebalck | TheJulia: good because? | 13:03 |
| TheJulia | because your configuration forces you down the right path, its a bug in ironic and CI | 13:06 |
| TheJulia | at least, I'm 99.95% convinced as such | 13:06 |
| *** dougsz has joined #openstack-ironic | 13:07 | |
| TheJulia | your hard coding local as your default boot option, cirros has boot headers embedded into the image to force an x86 boot | 13:08 |
| TheJulia | so the tests work in CI | 13:08 |
| TheJulia | ugh | 13:08 |
| * TheJulia ponders | 13:08 | |
| *** andrein has joined #openstack-ironic | 13:08 | |
| *** rh-jelabarre has quit IRC | 13:10 | |
| *** bobmel has quit IRC | 13:12 | |
| *** Goneri has joined #openstack-ironic | 13:20 | |
| * arne_wiebalck was reading some docs | 13:21 | |
| TheJulia | oO https://b1dbb34e22ddc2f6b152-1659b40e5c03e7f989419d6178d67ae8.ssl.cf1.rackcdn.com/679132/1/check/ironic-standalone/97c5492/controller/logs/ironic-bm-logs/node-0_no_ansi_2019-08-28-21:13:53_log.txt.gz | 13:23 |
| arne_wiebalck | Now, that's a RAID failure :-D | 13:24 |
| TheJulia | Well, first thing is likely first. We need to turn off the tempest tests :( | 13:29 |
| arne_wiebalck | The md restart during grub installation is failing ... since the device is already mounted? | 13:29 |
| TheJulia | looking to see | 13:29 |
| *** dsneddon has joined #openstack-ironic | 13:31 | |
| TheJulia | so looks like iscsi is still running | 13:31 |
| TheJulia | i that specific case | 13:32 |
| arne_wiebalck | ah | 13:32 |
| TheJulia | so likely a race I guess | 13:32 |
| arne_wiebalck | we could retry the restart, or check iscsi is gone, or find some replacement for my restart workaround | 13:34 |
| arne_wiebalck | maybe the recent rescan you introduced? | 13:35 |
| TheJulia | so the direct one gets further which is why there is so much confusion | 13:35 |
| TheJulia | https://b1dbb34e22ddc2f6b152-1659b40e5c03e7f989419d6178d67ae8.ssl.cf1.rackcdn.com/679132/1/check/ironic-standalone/97c5492/controller/logs/ironic-bm-logs/node-4_console_2019-08-28-21:20:55_log.txt.gz | 13:35 |
| *** priteau has quit IRC | 13:35 | |
| *** dsneddon has quit IRC | 13:36 | |
| TheJulia | perhaps, but I was just trying the same steps with the cirros image on a loopback, and I had no luck getting it to rescan properly | 13:36 |
| *** rachit7 has joined #openstack-ironic | 13:36 | |
| * arne_wiebalck has a meeting now, back in 20mins | 13:37 | |
| TheJulia | k | 13:37 |
| *** e0ne has joined #openstack-ironic | 13:40 | |
| *** rcernin_ has quit IRC | 13:41 | |
| *** e0ne has quit IRC | 13:43 | |
| openstackgerrit | Julia Kreger proposed openstack/ironic master: Fix get_boot_option logic https://review.opendev.org/679132 | 13:50 |
| openstackgerrit | Julia Kreger proposed openstack/ironic master: Disable software raid CI testing https://review.opendev.org/679257 | 13:50 |
| *** jcoufal has joined #openstack-ironic | 13:53 | |
| *** zbitter has quit IRC | 13:53 | |
| *** zbitter has joined #openstack-ironic | 13:54 | |
| *** rcernin_ has joined #openstack-ironic | 13:55 | |
| *** csmart has quit IRC | 13:55 | |
| openstackgerrit | Julia Kreger proposed openstack/ironic-tempest-plugin master: Make local boot explicit on software raid tests https://review.opendev.org/679258 | 13:56 |
| *** jawad_axd has quit IRC | 13:56 | |
| *** jawad_axd has joined #openstack-ironic | 13:57 | |
| TheJulia | so I think we do need to seriously consider adding code to IPA to grok if the contents written already contain boot loader data | 13:57 |
| *** jawad_axd has quit IRC | 13:57 | |
| *** dougsz has quit IRC | 13:57 | |
| TheJulia | In that case, we should be able to update the overall disk record for UEFI cases | 13:57 |
| TheJulia | honestly it is a bug fix for a regression, the direct interface + whole disks never really went down this path until fairly recently... and the code is linux specific | 13:58 |
| *** jawad_axd has joined #openstack-ironic | 13:58 | |
| *** jawad_ax_ has joined #openstack-ironic | 14:00 | |
| *** csmart has joined #openstack-ironic | 14:01 | |
| *** cdearborn has joined #openstack-ironic | 14:01 | |
| *** jawad_axd has quit IRC | 14:02 | |
| *** jawad_ax_ has quit IRC | 14:04 | |
| *** dsneddon has joined #openstack-ironic | 14:06 | |
| *** jawad_axd has joined #openstack-ironic | 14:09 | |
| *** dsneddon has quit IRC | 14:11 | |
| *** jawad_axd has quit IRC | 14:13 | |
| *** dsneddon has joined #openstack-ironic | 14:15 | |
| *** dougsz has joined #openstack-ironic | 14:15 | |
| arne_wiebalck | TheJulia: with "the contents already written" you mean the image data written to the install device, no? | 14:15 |
| TheJulia | Yes | 14:16 |
| arne_wiebalck | TheJulia: What would the IPA do if it detected a boot loader? | 14:16 |
| TheJulia | I think we would have to examine device contents to see if a MBR is present on the root of the device, the first partition, and see if a uefi boot partition exists. | 14:17 |
| *** rh-jelabarre has joined #openstack-ironic | 14:17 | |
| TheJulia | The only other reason to really run grub is the nvram update right? | 14:20 |
| dtantsur | ./stack.sh: line 1340: configure_neutron_after_post_config: command not found | 14:20 |
| dtantsur | WUT | 14:20 |
| * dtantsur assumes he's not cooking neutron right | 14:21 | |
| TheJulia | Ironic plugin right I think | 14:21 |
| dtantsur | for the context: that's the v6 metalsmith job | 14:21 |
| arne_wiebalck | TheJulia: and you would install grub if there is no MBR? | 14:21 |
| dtantsur | there's nothing fancy, I must be making some silly mistake | 14:22 |
| * arne_wiebalck is apparently slow | 14:22 | |
| *** dougsz has quit IRC | 14:22 | |
| *** e0ne has joined #openstack-ironic | 14:22 | |
| dtantsur | also, can someone please check my various IPA-b patches? https://review.opendev.org/#/q/project:openstack/ironic-python-agent-builder+status:open | 14:23 |
| dtantsur | we're getting excitingly close to a supported DIB image | 14:23 |
| arne_wiebalck | TheJulia: In case of a s/w RAID, the installation device is a partition on top of an md device which consists of partitions on a real disk. | 14:24 |
| dtantsur | TheJulia: (re scrollback) we haven't ever supported boot_loader:local in our CI, if that's your question. Cirros is incapable of that. | 14:24 |
| dtantsur | metalsmith CI builds its own centos image AND does not use tempest | 14:24 |
| dtantsur | kill cirros with fire can be an answer. using centos is not bad at all | 14:25 |
| dtantsur | relevant CI bits (we would want to cache the image) https://opendev.org/openstack/metalsmith/src/branch/master/playbooks/integration/centos-image.yaml | 14:25 |
| TheJulia | dtantsur: cirros actually has the loader already on it, the problem is our grub expectation | 14:27 |
| dtantsur | yep. this ^^ is why whole disk images work | 14:27 |
| *** mkrai has joined #openstack-ironic | 14:27 | |
| dtantsur | partition images + boot_option:local = sadness | 14:27 |
| TheJulia | cirros "partition" image local booting is bogus, the partition is empty | 14:28 |
| TheJulia | great sorrow | 14:28 |
| TheJulia | perhaps like romeo and juliet | 14:28 |
| dtantsur | :D | 14:29 |
| dtantsur | well, yes, it's never worked | 14:29 |
| * arne_wiebalck thinks of our last summit presentation where we had the Romeo + Juliet theme | 14:29 | |
| TheJulia | yeah.. the "copy ramdisk contents to disk upon ramdisk boot" thing is... ugh | 14:29 |
| TheJulia | What if we split out software raid standalone tests with something other than cirros? | 14:30 |
| arne_wiebalck | 16:27:57 <dtantsur> partition images + boot_option:local = sadness | 14:30 |
| arne_wiebalck | in general or in CI? | 14:30 |
| dtantsur | arne_wiebalck: with cirros | 14:30 |
| TheJulia | arne_wiebalck: CI with cirros | 14:30 |
| arne_wiebalck | ah, ok | 14:31 |
| dtantsur | works perfectly well with centos and other "real" operating systems | 14:31 |
| dtantsur | tripleo defaults to that | 14:31 |
| arne_wiebalck | ok, I was confused for a sec :) | 14:31 |
| dtantsur | TheJulia: sw raid uses whole disk images, no? | 14:31 |
| dtantsur | I mean, tests | 14:31 |
| TheJulia | dtantsur: yes, except you had a bug that was allowing your tests to silently kind of work with a default boot_option of netboot | 14:31 |
| dtantsur | that me! | 14:32 |
| dtantsur | :) | 14:32 |
| TheJulia | no, not your bug, just a long built bug in boot_option logic. When I started to tickle changing the default, I hit it | 14:32 |
| dtantsur | well, given that cirros is not (apparently) capable of booting from md volumes, I did not have many options | 14:32 |
| *** Lucas_Gray has quit IRC | 14:32 | |
| *** Wryhder has joined #openstack-ironic | 14:32 | |
| dtantsur | if we start using centos, we can have a normal job (I hope) | 14:32 |
| TheJulia | dtantsur: oh, but it is... just not capible of us trying to mount it, chroot it, and run commands against it | 14:33 |
| TheJulia | dtantsur: I guess if we can extend the OS_TEST_TIMEOUT.... | 14:33 |
| *** Wryhder is now known as Lucas_Gray | 14:33 | |
| TheJulia | the whole path of "everything uses grub, must get inside the image and run grub" is just... crazy | 14:33 |
| dtantsur | TheJulia: working on that: https://review.opendev.org/#/c/679242/ | 14:34 |
| patchbot | patch 679242 - tempest - Allow overridding OS_TEST_TIMEOUT via the role var... - 1 patch set | 14:34 |
| dtantsur | well... supporting partition images is kinda crazy | 14:34 |
| TheJulia | agreed | 14:34 |
| dtantsur | maybe not crazy, but too opinionated | 14:34 |
| dtantsur | everyone uses linux with grub kind of opinionated | 14:34 |
| *** Lucas_Gray has quit IRC | 14:35 | |
| *** dougsz has joined #openstack-ironic | 14:35 | |
| TheJulia | I _think_ the path for whole disk images should be "If iwdi and mbr_or_efi_part_present(root_dev) then skip to nvram metadata update as opposed to full bootloader config | 14:35 |
| TheJulia | and we should be able to set that nvram without needing grub in the image (but this is obviously something that I'm not going to get to until after september) | 14:36 |
| arne_wiebalck | are partition images widely used? (we do not, for instance, but use the very same whole disk images for physical (w/ and w/o s/w RAID) and virtual instances) | 14:36 |
| TheJulia | I would say it is kind of 50/50 | 14:37 |
| dtantsur | arne_wiebalck: I think many use them because they're used to | 14:37 |
| dtantsur | many use them because they don't know cloud-init can grow the last paritition | 14:37 |
| TheJulia | dtantsur: maybe put a "Stop using partition images" sign for your next conference talk? | 14:39 |
| dtantsur | in Stockholm? maybe :) | 14:39 |
| dtantsur | well, I guess we could start deprecating or at least discouraging them | 14:39 |
| dtantsur | (from our pov supporting two kinds of images is also not effortless) | 14:39 |
| TheJulia | that would be the longest deprecation cycle of ironic yet to date | 14:39 |
| dtantsur | ++ | 14:40 |
| dtantsur | and *somebody* will have to update tripleo :( | 14:40 |
| TheJulia | ugh | 14:40 |
| TheJulia | and build pipelines | 14:40 |
| TheJulia | maybe "please stop using partition images, they make us cry! And if you don't, we'll deprecate them!" | 14:41 |
| dtantsur | practically, it will only work the other way around | 14:41 |
| TheJulia | or "whenever you use a partition image, a cat goes without cuddles" | 14:41 |
| dtantsur | "hey, we've deprecated these. deal with it. or tell us why you hate us, we'll try to fix it" | 14:41 |
| TheJulia | ++ | 14:41 |
| arne_wiebalck | maybe "let us know why you prefer partition images over whole disk images" as a starter? | 14:42 |
| TheJulia | that is a really good point | 14:42 |
| dtantsur | arne_wiebalck: in an ideal world - yes. in the real one, people may not care to respond until we take actions | 14:42 |
| TheJulia | and perhaps a good question for me to pose as part of the project update | 14:42 |
| dtantsur | TheJulia: would you mind asking around in China? | 14:42 |
| * dtantsur writes China because he cannot spell the city, sorry | 14:42 | |
| arne_wiebalck | dtantsur: I agree | 14:42 |
| TheJulia | Shanghai | 14:42 |
| dtantsur | ++ | 14:42 |
| * TheJulia resists making awful jokes | 14:43 | |
| arne_wiebalck | it was just to avoid that we announce deprecation and then a use case comes along which needs partition images for whatever reason | 14:43 |
| arne_wiebalck | we can still ask with low expectations and proceed with deprecation ;) | 14:44 |
| TheJulia | "Please tell us why you prefer partition images over disk images... and if you don't we're going to deprecate partition images" | 14:44 |
| arne_wiebalck | TheJulia: ++ | 14:44 |
| TheJulia | "Failure to do so will make cats go without scritches and cuddles, and you don't want to make cats sad" | 14:44 |
| * arne_wiebalck sees the audience in Shanghai nod | 14:45 | |
| * TheJulia suspects cats will be invading the project update.... although she will need to see if that will be socially acceptable | 14:46 | |
| dtantsur | :D | 14:46 |
| dtantsur | but yes, it's a good idea | 14:46 |
| arne_wiebalck | FWIW and to build up confidence: yesterday and today we set up more than 400 nodes with s/w RAID (upstream code backported to Stein) and created instances on all of them | 14:48 |
| dtantsur | awesome | 14:50 |
| dtantsur | software RAID is arguable one of the most awaited features in the ironic history | 14:51 |
| dtantsur | (I still think we must implement disk hits asap) | 14:51 |
| arne_wiebalck | dtantsur: physical disk selection you mean? | 14:51 |
| dtantsur | yep | 14:51 |
| arne_wiebalck | yes | 14:51 |
| * dtantsur should have written "disk hints", but "n" ran away | 14:52 | |
| arne_wiebalck | we have quite some nodes being added these weeks and months, but I'd of course be interested to work on this | 14:52 |
| TheJulia | arne_wiebalck: that is awesome news to hear | 14:55 |
| arne_wiebalck | TheJulia: and is 100% upstream code | 14:56 |
| arne_wiebalck | TheJulia: the only thing I did was to include the clean steps so that s/w RAID gets created during cleaning :-P | 14:57 |
| arne_wiebalck | TheJulia: include in our h/w manager | 14:57 |
| arne_wiebalck | TheJulia: during automatic cleaning | 14:57 |
| * arne_wiebalck starts to write sentences which are hard to understand | 14:58 | |
| *** dougsz has quit IRC | 14:58 | |
| TheJulia | :) | 14:58 |
| *** mkrai has quit IRC | 15:00 | |
| *** mkrai has joined #openstack-ironic | 15:02 | |
| *** jawad_axd has joined #openstack-ironic | 15:11 | |
| *** e0ne has quit IRC | 15:14 | |
| *** jawad_axd has quit IRC | 15:16 | |
| *** mkrai has quit IRC | 15:18 | |
| *** dsneddon has quit IRC | 15:18 | |
| *** mkrai has joined #openstack-ironic | 15:21 | |
| *** sthussey has joined #openstack-ironic | 15:22 | |
| *** jtwill98 has joined #openstack-ironic | 15:44 | |
| *** dsneddon has joined #openstack-ironic | 15:48 | |
| *** mkrai has quit IRC | 15:50 | |
| *** mkrai has joined #openstack-ironic | 15:50 | |
| *** trandles has joined #openstack-ironic | 15:52 | |
| *** dsneddon has quit IRC | 15:54 | |
| *** mkrai has quit IRC | 15:56 | |
| NobodyCam | Good morning Ironic folks | 15:57 |
| TheJulia | good morning NobodyCam | 15:58 |
| *** lucasagomes has quit IRC | 15:59 | |
| NobodyCam | Good Morning TheJulia :) | 15:59 |
| NobodyCam | ready to come back to the west coast? | 16:00 |
| *** gyee has joined #openstack-ironic | 16:00 | |
| *** jcoufal has quit IRC | 16:03 | |
| dtantsur | morning NobodyCam! | 16:04 |
| NobodyCam | Morning dtantsur :) | 16:04 |
| *** rachit7 has quit IRC | 16:09 | |
| *** dougsz has joined #openstack-ironic | 16:14 | |
| arne_wiebalck | bye everyone o/ | 16:18 |
| dtantsur | c u arne_wiebalck | 16:19 |
| NobodyCam | night arne_wiebalck | 16:24 |
| *** trandles has quit IRC | 16:25 | |
| *** dsneddon has joined #openstack-ironic | 16:25 | |
| *** dsneddon has quit IRC | 16:30 | |
| *** igordc has joined #openstack-ironic | 16:44 | |
| TheJulia | NobodyCam: very much so. Saw on one of the cameras that the wind has calmed down | 16:44 |
| *** alexmcleod has quit IRC | 16:45 | |
| *** andrein has quit IRC | 16:47 | |
| *** tesseract has quit IRC | 16:50 | |
| dtantsur | TheJulia, NobodyCam https://review.opendev.org/#/c/679058/ | 16:52 |
| patchbot | patch 679058 - bifrost - Stop using deprecated OS_URL - 1 patch set | 16:52 |
| dtantsur | so bifrost is not broken, it was probably a glitch | 16:52 |
| *** dougsz has quit IRC | 16:56 | |
| *** dsneddon has joined #openstack-ironic | 16:59 | |
| *** derekh has quit IRC | 16:59 | |
| *** derekh has joined #openstack-ironic | 16:59 | |
| *** e0ne has joined #openstack-ironic | 17:02 | |
| *** ijw has joined #openstack-ironic | 17:02 | |
| *** dtantsur is now known as dtantsur|afk | 17:03 | |
| dtantsur|afk | see you tomorrow | 17:03 |
| *** e0ne has quit IRC | 17:04 | |
| *** e0ne has joined #openstack-ironic | 17:04 | |
| *** dsneddon has quit IRC | 17:04 | |
| *** e0ne has quit IRC | 17:04 | |
| *** derekh has quit IRC | 17:09 | |
| *** jawad_axd has joined #openstack-ironic | 17:26 | |
| *** jawad_axd has quit IRC | 17:29 | |
| *** dsneddon has joined #openstack-ironic | 17:29 | |
| *** andrein has joined #openstack-ironic | 17:32 | |
| *** dsneddon has quit IRC | 17:35 | |
| *** e0ne has joined #openstack-ironic | 17:35 | |
| *** e0ne has quit IRC | 17:35 | |
| *** jtomasek has quit IRC | 17:36 | |
| *** jtomasek has joined #openstack-ironic | 17:46 | |
| *** ricolin has quit IRC | 17:55 | |
| *** andrein has quit IRC | 17:57 | |
| *** dsneddon has joined #openstack-ironic | 18:06 | |
| *** jtomasek has quit IRC | 18:08 | |
| *** dsneddon has quit IRC | 18:11 | |
| *** jtomasek has joined #openstack-ironic | 18:14 | |
| *** dsneddon has joined #openstack-ironic | 18:18 | |
| *** ijw has quit IRC | 18:19 | |
| *** ijw has joined #openstack-ironic | 18:19 | |
| *** ijw has quit IRC | 18:21 | |
| *** ijw has joined #openstack-ironic | 18:22 | |
| *** ijw has quit IRC | 18:23 | |
| *** ijw has joined #openstack-ironic | 18:24 | |
| *** ijw has quit IRC | 18:26 | |
| *** ijw has joined #openstack-ironic | 18:27 | |
| *** ijw_ has joined #openstack-ironic | 18:28 | |
| *** e0ne has joined #openstack-ironic | 18:29 | |
| *** ijw_ has quit IRC | 18:29 | |
| *** ijw has quit IRC | 18:32 | |
| *** ijw_ has joined #openstack-ironic | 18:32 | |
| *** ijw_ has quit IRC | 18:32 | |
| *** e0ne has quit IRC | 18:32 | |
| *** andrein has joined #openstack-ironic | 18:34 | |
| *** ijw has joined #openstack-ironic | 18:35 | |
| *** ijw_ has joined #openstack-ironic | 18:36 | |
| *** ijw_ has quit IRC | 18:37 | |
| *** ijw has quit IRC | 18:38 | |
| *** ijw has joined #openstack-ironic | 18:39 | |
| *** ijw has quit IRC | 18:43 | |
| *** jtomasek has quit IRC | 18:54 | |
| *** e0ne has joined #openstack-ironic | 18:55 | |
| *** e0ne has quit IRC | 19:01 | |
| *** ijw has joined #openstack-ironic | 19:01 | |
| *** e0ne has joined #openstack-ironic | 19:03 | |
| *** ijw has quit IRC | 19:04 | |
| *** e0ne has quit IRC | 19:04 | |
| *** e0ne has joined #openstack-ironic | 19:05 | |
| *** ijw has joined #openstack-ironic | 19:05 | |
| *** e0ne has quit IRC | 19:05 | |
| *** Nisha_away has joined #openstack-ironic | 19:44 | |
| openstackgerrit | Nisha Agarwal proposed openstack/ironic master: RPC changes for Graphical Console https://review.opendev.org/640827 | 19:59 |
| *** Nisha_away has quit IRC | 20:05 | |
| openstackgerrit | Nisha Brahmankar proposed openstack/ironic-python-agent master: Adds CI job that uses DIB build images https://review.opendev.org/676239 | 20:18 |
| *** ijw_ has joined #openstack-ironic | 20:20 | |
| *** ijw has quit IRC | 20:24 | |
| openstackgerrit | Julia Kreger proposed openstack/ironic-python-agent master: Attempt tgtd session detach https://review.opendev.org/679332 | 20:25 |
| openstackgerrit | Julia Kreger proposed openstack/ironic-python-agent master: Don't try to put a bootloader in place when bootable https://review.opendev.org/679333 | 20:25 |
| TheJulia | dtantsur|afk: ^^^^^^ Needs a reno, but should cause bootloader to be bypassed if the partition boot flag is present. We likely need to actually examine contents... but... *shrug* | 20:27 |
| *** Goneri has quit IRC | 20:35 | |
| *** trandles has joined #openstack-ironic | 20:35 | |
| trandles | Ok, today's newb question. I added image_source=file:///images/centos7.qcow2 and it's freaking out the conductor when trying to validate. | 20:39 |
| trandles | | deploy | False | Unexpected exception, traceback saved into log by ironic conductor service that is running on do-sn1.localdomain: 'NoneType' object has no attribute 'rstrip' | | 20:39 |
| trandles | 2019-08-29 14:38:07.306 105149 ERROR ironic.conductor.manager [req-ef8b445d-3704-43da-a947-733f18e6418c - - - - -] Unexpected exception occurred while validating deploy driver interface for driver ipmi: 'NoneType' object has no attribute 'rstrip' on node 2f9fd223-839c-4bf8-ae22-fc8d7394c786.: AttributeError: 'NoneType' object has no attribute 'rstrip' | 20:39 |
| trandles | 2019-08-29 14:38:07.306 105149 ERROR ironic.conductor.manager Traceback (most recent call last): | 20:39 |
| trandles | 2019-08-29 14:38:07.306 105149 ERROR ironic.conductor.manager File "/usr/lib/python2.7/site-packages/ironic/conductor/manager.py", line 2206, in validate_driver_interfaces | 20:39 |
| trandles | 2019-08-29 14:38:07.306 105149 ERROR ironic.conductor.manager iface.validate(task) | 20:39 |
| trandles | 2019-08-29 14:38:07.306 105149 ERROR ironic.conductor.manager File "/usr/lib/python2.7/site-packages/ironic_lib/metrics.py", line 60, in wrapped | 20:39 |
| trandles | 2019-08-29 14:38:07.306 105149 ERROR ironic.conductor.manager result = f(*args, **kwargs) | 20:39 |
| trandles | 2019-08-29 14:38:07.306 105149 ERROR ironic.conductor.manager File "/usr/lib/python2.7/site-packages/ironic/drivers/modules/iscsi_deploy.py", line 394, in validate | 20:39 |
| trandles | 2019-08-29 14:38:07.306 105149 ERROR ironic.conductor.manager validate(task) | 20:39 |
| trandles | 2019-08-29 14:38:07.306 105149 ERROR ironic.conductor.manager File "/usr/lib/python2.7/site-packages/ironic_lib/metrics.py", line 60, in wrapped | 20:39 |
| trandles | 2019-08-29 14:38:07.306 105149 ERROR ironic.conductor.manager result = f(*args, **kwargs) | 20:39 |
| trandles | 2019-08-29 14:38:07.306 105149 ERROR ironic.conductor.manager File "/usr/lib/python2.7/site-packages/ironic/drivers/modules/iscsi_deploy.py", line 320, in validate | 20:40 |
| trandles | 2019-08-29 14:38:07.306 105149 ERROR ironic.conductor.manager deploy_utils.get_ironic_api_url() | 20:40 |
| trandles | 2019-08-29 14:38:07.306 105149 ERROR ironic.conductor.manager File "/usr/lib/python2.7/site-packages/ironic/drivers/modules/deploy_utils.py", line 123, in get_ironic_api_url | 20:40 |
| trandles | 2019-08-29 14:38:07.306 105149 ERROR ironic.conductor.manager ironic_api = ironic_api.rstrip('/') | 20:40 |
| trandles | 2019-08-29 14:38:07.306 105149 ERROR ironic.conductor.manager AttributeError: 'NoneType' object has no attribute 'rstrip' | 20:40 |
| trandles | 2019-08-29 14:38:07.306 105149 ERROR ironic.conductor.manager | 20:40 |
| trandles | I'm using iscsi deploy interface | 20:40 |
| * TheJulia blinks | 20:42 | |
| TheJulia | hmm | 20:43 |
| TheJulia | trandles: i guess your not using keystone | 20:44 |
| trandles | I can provide more output if it'll help...but I'm not sure and I feel like a backtrace is already abusing the channel | 20:44 |
| TheJulia | and that conductor.api_url is not set | 20:44 |
| trandles | Totally standalone | 20:45 |
| TheJulia | that setting needs to be set them | 20:45 |
| TheJulia | [conductor]api_url | 20:45 |
| trandles | Even if I'm using json-rpc? | 20:45 |
| TheJulia | yes, that url actually gets used by the conductor to tell the agent where to call home | 20:45 |
| TheJulia | which is why it can be different from the API's perception, or normally keystone... | 20:46 |
| trandles | ahok | 20:46 |
| trandles | Ah _space_ ok | 20:46 |
| trandles | \o/ fixed, thanks TheJulia | 20:49 |
| openstackgerrit | Julia Kreger proposed openstack/ironic master: Provide a more useful error message https://review.opendev.org/679343 | 20:49 |
| trandles | Says that setting is deprecated in ironic.conf...not sure what you'll do for standalone in the future | 20:49 |
| TheJulia | ugh | 20:49 |
| * TheJulia ponders "un-deprecating" | 20:50 | |
| *** rh-jelabarre has quit IRC | 20:57 | |
| trandles | Ah, it looks like maybe I should use [conductor]endpoint_override instead. That's the hint given in the deprecated note. It exists for other services (like [glance], [cinder]) but not for [conductor]. I'm going to try it in that section and see if it works. | 20:57 |
| trandles | That worked, actually [service_catalog]endpoint_override is the right value | 21:01 |
| trandles | So continue deprecating | 21:01 |
| *** ijw_ has quit IRC | 21:02 | |
| *** ijw has joined #openstack-ironic | 21:07 | |
| TheJulia | \o/ | 21:07 |
| TheJulia | More to fix then :\ | 21:07 |
| *** rcernin_ has quit IRC | 21:25 | |
| *** trandles has left #openstack-ironic | 22:05 | |
| *** ijw has quit IRC | 22:18 | |
| *** ijw has joined #openstack-ironic | 22:34 | |
| *** ijw has quit IRC | 22:37 | |
| *** ijw has joined #openstack-ironic | 22:38 | |
| *** rcernin has joined #openstack-ironic | 22:43 | |
| *** cdearborn has quit IRC | 23:37 | |
| *** andrein has quit IRC | 23:41 | |
| *** mrda has quit IRC | 23:49 | |
| *** mrda has joined #openstack-ironic | 23:50 | |
| *** sthussey has quit IRC | 23:52 | |
| *** hjensas has quit IRC | 23:52 | |
| *** gyee has quit IRC | 23:55 | |
| *** hjensas has joined #openstack-ironic | 23:56 | |
Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!