Tuesday, 2020-08-11

*** diurnalist has quit IRC00:06
*** diurnalist has joined #openstack-ironic00:31
*** stevebaker has joined #openstack-ironic00:32
*** rloo has quit IRC00:46
*** diurnalist has quit IRC00:51
*** diurnalist has joined #openstack-ironic00:53
*** diurnalist has quit IRC00:59
*** diurnalist has joined #openstack-ironic01:11
*** diurnalist has quit IRC01:47
*** diurnalist has joined #openstack-ironic02:10
*** gyee has quit IRC02:11
*** diurnalist has quit IRC02:14
*** rh-jelabarre has quit IRC02:16
*** rcernin has quit IRC02:37
*** rcernin has joined #openstack-ironic02:48
*** mraineri has quit IRC03:04
*** baha has quit IRC03:04
*** bnemec has quit IRC03:04
*** mkrai has joined #openstack-ironic03:04
*** irclogbot_2 has quit IRC03:05
*** baha has joined #openstack-ironic03:05
*** bnemec has joined #openstack-ironic03:06
*** irclogbot_2 has joined #openstack-ironic03:07
*** diurnalist has joined #openstack-ironic03:23
*** yaawang_ has quit IRC03:48
*** yaawang_ has joined #openstack-ironic03:49
*** rcernin has quit IRC03:54
*** diurnalist has quit IRC04:02
*** cdearborn has quit IRC04:11
*** tzumainn has quit IRC04:11
*** diurnalist has joined #openstack-ironic04:13
*** rcernin has joined #openstack-ironic04:21
*** abdysn has joined #openstack-ironic05:21
*** belmoreira has joined #openstack-ironic05:56
*** belmoreira has quit IRC05:58
*** belmoreira has joined #openstack-ironic06:14
*** diurnalist has quit IRC06:14
rpittaugood morning ironic! o/06:45
*** Abdallahyas has joined #openstack-ironic06:56
*** abdysn has quit IRC06:59
*** mkrai has quit IRC06:59
*** Lucas_Gray has joined #openstack-ironic07:12
*** ntt__ has joined #openstack-ironic07:15
rpittauwhoooo virtual PTG registration are open :)07:15
*** dougsz has joined #openstack-ironic07:30
*** yedongcan has joined #openstack-ironic07:33
iurygregorygood morning rpittau and Ironic o/07:34
rpittauhey iurygregory :)07:35
*** tosky has joined #openstack-ironic07:41
*** Wryhder has joined #openstack-ironic07:44
*** Lucas_Gray has quit IRC07:45
*** Wryhder is now known as Lucas_Gray07:45
*** Abdallahyas has quit IRC07:54
iurygregoryouch, we have a test that does run parted --version and since I switched to use sudo everything blows up, any ideas?07:55
*** abdysn has joined #openstack-ironic07:55
*** ntt has joined #openstack-ironic07:56
*** ntt__ has quit IRC07:58
iurygregoryhttps://review.opendev.org/#/c/745535/1/ironic_lib/tests/test_disk_utils.py this is the test .-.07:58
patchbotpatch 745535 - ironic-lib - Refactor ironic-lib to move to oslo.privsep - 1 patch set07:58
jandersgood morning rpittau  iurygregory and Ironic o/08:09
rpittauhey janders :)08:09
iurygregoryjanders, o/08:09
*** yedongcan has quit IRC08:10
*** lucasagomes has joined #openstack-ironic08:13
*** ntt_ has joined #openstack-ironic08:21
*** ntt has quit IRC08:24
*** priteau has joined #openstack-ironic08:26
*** priteau has quit IRC08:42
gudrutis0Hello ironic o/08:43
iurygregoryhey gudrutis008:44
jandersg'day gudrutis0 o/08:53
*** priteau has joined #openstack-ironic08:58
openstackgerritRiccardo Pittau proposed openstack/ironic master: Enforce autospec in irmc tests  https://review.opendev.org/74564209:00
*** hjensas has quit IRC09:06
*** hjensas has joined #openstack-ironic09:06
*** rcernin has quit IRC09:11
*** rcernin has joined #openstack-ironic09:34
*** tkajinam has quit IRC09:36
openstackgerritRiccardo Pittau proposed openstack/ironic master: [WIP] Replace retrying with tenacity  https://review.opendev.org/37657409:40
*** Lucas_Gray has quit IRC09:49
*** rcernin has quit IRC09:56
openstackgerritRiccardo Pittau proposed openstack/ironic master: [DNM] test ci  https://review.opendev.org/74488810:07
*** priteau has quit IRC10:11
*** yaawang_ has quit IRC10:13
openstackgerritMerged openstack/ironic master: Use property plus abstractmethod for abstractproperty  https://review.opendev.org/74494510:40
openstackgerritVerification of a change to openstack/ironic failed: Extend base build timeouts  https://review.opendev.org/74534811:01
*** hipolito has joined #openstack-ironic11:03
hipolitoHi, I'm unable to build an IPA image based on openSuse with ironic-python-agent-builder11:07
hipolitoI tried all combinations of opensuse, opensuse-minimal and release versions11:07
hipolitoit always fails on "Running command: ['install-packages', '-e', 'kbd', 'cloud-init', 'rsync', 'git', 'man-db']"11:08
hipolitothat always finishes with "returncode: 104", "install-packages failed with returncode 104"11:08
hipolitois the opensuse target supported at all? it is mentioned that it is experimental11:09
janderswrapping up for the evening, see you tomorrow Ironic o/11:29
*** k_mouza has joined #openstack-ironic11:29
*** k_mouza has quit IRC11:30
*** rcernin has joined #openstack-ironic11:31
*** k_mouza has joined #openstack-ironic11:33
*** Lucas_Gray has joined #openstack-ironic11:40
gudrutis0bye janders11:51
*** rh-jelabarre has joined #openstack-ironic11:53
openstackgerritMerged openstack/ironic stable/stein: Remove locks before RPC bus is started  https://review.opendev.org/74381012:08
openstackgerritMerged openstack/ironic stable/queens: Remove locks before RPC bus is started  https://review.opendev.org/74381212:08
*** k_mouza has quit IRC12:11
*** iurygregory has quit IRC12:25
openstackgerritRiccardo Pittau proposed openstack/ironic master: Enforce autospec in irmc tests  https://review.opendev.org/74564212:26
*** iurygregory has joined #openstack-ironic12:26
openstackgerritZygimantas Matonis proposed openstack/ironic-inspector master: Log outcome of `check_conditions` method.  https://review.opendev.org/74566312:34
openstackgerritRiccardo Pittau proposed openstack/ironic-python-agent master: Use ubuntu focal for ipa src  https://review.opendev.org/72579412:52
*** JamesBenson has joined #openstack-ironic12:57
openstackgerritRiccardo Pittau proposed openstack/ironic master: [WIP] Replace retrying with tenacity  https://review.opendev.org/37657413:03
gudrutis0Hey guys, where did the  source of https://opendev.org/openstack/ironic-inspector/src/branch/master/doc/source/user/http-api.rst go?13:06
TheJuliahipolito: There have been recent reports of issues with opensuse and diskimage builder, and also bifrost. It may help if you are able to somehow post more of the output13:06
gudrutis0TheJulia: hi o/13:07
openstackgerritRiccardo Pittau proposed openstack/ironic master: [WIP] Replace retrying with tenacity  https://review.opendev.org/37657413:07
rpittaugudrutis0: https://docs.openstack.org/api-ref/baremetal-introspection/13:07
iurygregorygudrutis0, if you open the file you will have the url =)13:08
TheJuliagood morning everyone13:08
rpittaugood morning TheJulia :)13:08
iurygregorygood morning TheJulia =)13:08
gudrutis0yes, I see that, but I am looking for the source13:08
*** rloo has joined #openstack-ironic13:09
iurygregoryyou mean the patch that moved the things or the code of the api?13:09
rpittaugudrutis0: https://opendev.org/openstack/ironic-inspector/src/branch/master/api-ref/source13:09
gudrutis0I was looking where to put this change13:15
gudrutis0https://review.opendev.org/#/c/731794/7/doc/source/user/http-api.rst13:15
patchbotpatch 731794 - ironic-inspector - More flexible definitions of conditions in rules - 7 patch sets13:15
openstackgerritRiccardo Pittau proposed openstack/ironic-inspector master: Replace retrying with tenacity  https://review.opendev.org/74566713:16
gudrutis0ok found it api-ref/source/introspection-api-versions.inc13:16
rpittaugudrutis0: https://opendev.org/openstack/ironic-inspector/src/branch/master/api-ref/source/introspection-api-versions.inc13:17
rpittauyeah there :)13:17
openstackgerritRiccardo Pittau proposed openstack/ironic-inspector master: Replace retrying with tenacity  https://review.opendev.org/74566713:18
openstackgerritRiccardo Pittau proposed openstack/ironic-inspector master: Replace retrying with tenacity  https://review.opendev.org/74566713:19
*** dougsz has quit IRC13:32
*** dougsz has joined #openstack-ironic13:34
*** billp has joined #openstack-ironic13:38
*** tzumainn has joined #openstack-ironic13:43
rpittausuddenly... rain13:44
*** ociuhandu_ has joined #openstack-ironic13:46
*** mkrai has joined #openstack-ironic13:47
*** ociuhandu has quit IRC13:47
TheJuliarain in general can be a good thing13:58
TheJuliaWe've not had any rain so far thi syear13:58
TheJuliaBut, desert13:58
TheJuliahttps://storyboard.openstack.org/#!/story/2008001 <-- dsneddon, bfournie (if your interested in lacp headaches)13:59
*** abdysn has quit IRC14:03
*** rcernin has quit IRC14:08
*** rcernin has joined #openstack-ironic14:19
*** rcernin has quit IRC14:24
*** rcernin has joined #openstack-ironic14:27
hipolitoTheJulia: Sorry I was out for a bit. Here's a complete output https://pastebin.com/Rf7k7K2Q14:30
*** rcernin has quit IRC14:31
hipolitoI also have problems building centos 8 IPA: https://pastebin.com/7XZEFaEU14:38
*** diurnalist has joined #openstack-ironic14:40
iurygregoryTheJulia, do we still need to verify this https://github.com/openstack/ironic-lib/blob/e4e11ccc7df5cf566211e3b347067ca196691d54/ironic_lib/tests/test_disk_utils.py#L872-L876 ?14:42
rpittauhipolito: can you please try with centos and not centos-minimal ?14:42
*** livelace has joined #openstack-ironic14:44
rpittauhipolito: nvm, sorry, I think it's just missing libffi-devel, which version of ipa-builder are you using ?14:47
hipolitorpittau: sure, it's doing it now14:47
hipolitooh, it's IPA from PIP, and I have installed libffi-devel on the build machine14:48
rpittauhipolito: ok, so you installed the last version from pip ?14:48
hipolitorpittau: yes, I just run the pip command exactly from the instruction page14:49
hipolitorpittau: on centos I had to change it to pip3 though14:49
rpittauhipolito: cool, I guess it's version 2.0.0, can you confirm it ? you should be able to check using 'pip freeze'14:50
hipolito"ironic-python-agent-builder==2.0.0" "diskimage-builder==3.1.0"14:51
rpittauhipolito: cool, that's a bug and it was fixed in master but not in that release14:51
rpittauTheJulia: I think we need to release ipa-builder :)14:52
hipolitoalright, I will try with master14:52
hipolitocan I "pip install" the git repo?14:52
rpittauhipolito: thanks14:52
rpittauhipolito: you can use pip to install with git, yes14:52
*** livelace has quit IRC15:01
hipolitoripttau: I'm running both centos 8 and opensuse with ipa-builder master. Wish me luck :D15:03
hipolitoopensuse still broken, same error15:06
openstackgerritRiccardo Pittau proposed openstack/ironic-inspector master: Fix requirements job  https://review.opendev.org/74569515:07
*** billp has quit IRC15:07
*** mkrai has quit IRC15:07
rpittauhipolito: alright, for suse I'm not sure we can do much as we don't support it with ipa-builder, but you can always propose patches for that15:09
TheJuliarpittau: I guess so, can you propose the change and I can +1 it15:12
rpittauTheJulia: it's there :)15:13
TheJulia\o/15:13
TheJuliahjensas: https://review.opendev.org/74569815:13
patchbotpatch 745698 - diskimage-builder - Handle NetworkManager for dhcp-all-interfaces - 1 patch set15:13
openstackgerritJulia Kreger proposed openstack/ironic-python-agent master: empty magical test change  https://review.opendev.org/74570015:21
TheJuliahjensas: ^^^ should test the above dib chnage15:23
hipolitorpittau: I got centos working! Thanks a lot!15:23
rpittauhipolito: yup, thanks for reporting the issue :)15:23
hipolitorpittau: I'll try to make time to get to opensuse :D15:23
rpittauhipolito: good luck! :)15:23
*** belmoreira has quit IRC15:25
*** hipolito has quit IRC15:34
*** mkrai has joined #openstack-ironic15:37
*** rcernin has joined #openstack-ironic15:40
*** rcernin has quit IRC15:44
*** bdodd has joined #openstack-ironic15:49
openstackgerritIury Gregory Melo Ferreira proposed openstack/ironic-lib master: Add initial privsep structure  https://review.opendev.org/74553615:49
openstackgerritRiccardo Pittau proposed openstack/ironic-lib master: Use tenacity to retry convert_image  https://review.opendev.org/74570415:52
TheJuliarpittau: is tenancy in the global requirements going back in time?15:53
rpittauTheJulia: yeah, tenacity is there at least since 2 years ago15:55
rpittauactually, I tracked it in newton, so even longer15:56
TheJuliaack15:57
openstackgerritIury Gregory Melo Ferreira proposed openstack/ironic master: Init privsep  https://review.opendev.org/74570616:01
*** rloo has quit IRC16:02
*** rloo has joined #openstack-ironic16:03
*** dking has quit IRC16:04
*** lucasagomes has quit IRC16:04
*** ntt_ has quit IRC16:29
*** Lucas_Gray has quit IRC16:29
*** Lucas_Gray has joined #openstack-ironic16:31
*** dougsz has quit IRC16:35
*** zzzeek has quit IRC16:47
openstackgerritRiccardo Pittau proposed openstack/ironic master: [WIP] Replace retrying with tenacity  https://review.opendev.org/37657416:51
rpittauo/16:52
*** naved001 has quit IRC16:55
*** naved001 has joined #openstack-ironic16:55
openstackgerritIury Gregory Melo Ferreira proposed openstack/ironic master: Init privsep  https://review.opendev.org/74570616:58
*** tosky has quit IRC16:59
*** k_mouza has joined #openstack-ironic17:04
*** k_mouza has quit IRC17:09
*** JamesBen_ has joined #openstack-ironic17:14
*** JamesBenson has quit IRC17:14
*** mkrai has quit IRC17:19
*** zbitter has joined #openstack-ironic17:25
*** zaneb has quit IRC17:25
*** Lucas_Gray has quit IRC17:26
*** diurnalist has quit IRC18:15
*** k_mouza has joined #openstack-ironic18:23
*** trandles has joined #openstack-ironic18:55
*** k_mouza has quit IRC19:20
*** k_mouza has joined #openstack-ironic19:26
*** k_mouza has quit IRC19:26
*** k_mouza has joined #openstack-ironic19:27
*** k_mouza has quit IRC19:32
trandlesHi all. Inspector-related question. I'm seeing a traceback from IPA during inspection that I haven't been able to search up much help for: "No conductor service registered which supports driver agent_ipmitool for conductor group ""." Is this complaint about ironic-conductor or ironic-inspector-conductor?19:38
trandlesI'm on Train, installed via RH RPM, ironic-inspector running in standalone, ironic running in standalone.19:39
*** rcernin has joined #openstack-ironic19:41
iurygregoryseems like your ironic.conf is not configured correct19:41
iurygregoryalso agent_ipmitool is a classic driver so you need to have ipmi configuration set19:42
trandlesI was able to deploy fine using ironic. It was only when I started trying to auto-enroll using inspector that this error started.19:43
trandlesRE: agent_ipmitool...that doesn't appear in ironic.conf or inspector.conf, so that's part of why I'm confused.19:44
trandlesI have no idea where "agent_ipmitool" is coming from19:44
*** rcernin has quit IRC19:45
iurygregoryseems like the auto-enroll is trying to use agent_ipmitool19:47
iurygregoryhttps://github.com/openstack/ironic/blob/stable/train/doc/source/admin/upgrade-to-hardware-types.rst shows a bit how the config should look like19:49
trandlesthat matches what I have19:56
trandlesI'll keep looking19:56
trandles~]$ sudo grep ipmi /etc/ironic/ironic.conf19:56
trandlesenabled_hardware_types = ipmi19:56
trandlesenabled_console_interfaces = ipmitool-socat,no-console19:56
trandlesenabled_management_interfaces = ipmitool19:56
trandlesenabled_power_interfaces = ipmitool19:56
trandles#enabled_vendor_interfaces = ipmitool,no-vendor19:56
trandles[ipmi]19:56
trandles# Kill `ipmitool` process invoked by ironic to read node power19:56
trandles# state if `ipmitool` process does not exit after19:56
trandles# 'ipmi_disable_boot_timeout' option in node's 'driver_info'19:56
trandles# Additional errors ipmitool may encounter, specific to the19:56
iurygregorydon't paste the full config here19:56
trandles#additional_retryable_ipmi_errors =19:57
trandles# ipmitool - IPMItool19:57
trandles#sensor_method = ipmitool19:57
iurygregoryuse a paste service19:57
iurygregory=)19:57
iurygregorypaste.openstack.org19:57
trandlesah, thx for the pointer19:57
iurygregoryno worries =)19:57
trandleshttp://paste.openstack.org/show/796746/19:58
*** gyee has joined #openstack-ironic19:59
openstackgerritJulia Kreger proposed openstack/ironic master: Handle an older agent with agent_token  https://review.opendev.org/74573620:15
TheJuliabfournie: ^^^20:16
TheJuliao/ trandles20:16
trandleshello TheJulia20:16
TheJuliaagent_ipmitool is likely coming form inspector config20:17
trandlesmy paste link above greps for "ipmi" in inspector.conf :/20:17
iurygregoryyeah the config seems correct20:18
TheJuliahmmmm20:18
TheJuliatrandles: what version are you running?20:18
iurygregoryenroll_node_driver is set to ipmi20:18
trandlesI was in the middle of debugging this when my team leader "gently nudged" me with a phone call on my home phone saying "why aren't you in this meeting?"20:18
trandlesI might be slow to respond here :(20:18
TheJuliano worries20:18
iurygregoryTheJulia, trandles is using Train20:19
TheJuliaI should have known this... doh20:19
trandlesI _think_ it's Train although it's installed via RPM20:19
TheJuliaoO20:20
trandlesso built by RDO20:20
trandlesRPM version is 9.3.020:20
trandlesmight be quite old by now, maybe I should get off the RPMs and start installing in some other way20:20
TheJuliabravo rdo20:21
TheJuliawe never cut a 9.3.0 release20:21
trandlesit's just a lot easier to deal with stuff at work if I'm using "trusted, signed" packages :/20:21
TheJuliaI'm guessing 9.2.3+20:21
iurygregorywut20:21
trandlesRelease     : 0.20191217011302.b2c0c52.el720:21
trandlesRelease     : 0.20191217011302.b2c0c52.el720:21
trandlesRelease     : 0.20191217011302.b2c0c52.el720:21
trandlesRelease: 0.20191217011302.b2c0c52.el720:21
trandlesWTH...20:22
trandlessorry20:22
trandles^ this sums up my day :P20:22
TheJuliaheh, no worries20:22
TheJuliawell this is weird... the default is fake-hardware20:24
TheJuliatrandles: has inspector been restarted recently?20:24
TheJulianone of this config is mutable so it would need a restart if ther ehas been any config changes20:24
trandlesa whole lot of times today while I was messing with config20:25
TheJuliayou sure you don't have an older inspector running someplace?20:38
trandlesLOL that _would_ be the cherry on top of today ;)20:39
trandlesaccording to ps, and/or systemd, in combination with `rpm -qf` it's from the RPM20:39
TheJuliacould it be next to a novell machine that got drywalled off from the world?20:39
trandlesNetWare!20:40
trandlesI thought I was using PXE, turns out it's IPX ;)20:40
TheJulialol20:40
trandlesI did have debug enabled for a time today, scouring the logs to see what it reports for config values20:42
TheJuliait should be fake-hardware at worst20:42
trandlesoh, "processing.node_not_found_hook = enroll"  < - could this be it?20:44
TheJuliahttps://github.com/openstack/ironic-inspector/blob/master/setup.cfg#L5320:45
trandlesoooh, ok, that's _not_ it20:45
TheJuliahttps://github.com/openstack/ironic-inspector/blob/master/ironic_inspector/plugins/discovery.py#L7620:46
trandlesfrom the debug logs: discovery.enroll_node_driver   = ipmi20:46
TheJuliais there more than one file in /etc/ironic-inspector ?20:50
trandlesyeah, the only ones appearing on ironic-inspector's command line are inspector.conf and inspector-dist.conf, but -dist.conf is just log_dir, state_dir, use_stderr20:52
*** openstackgerrit has quit IRC20:52
trandlesplus the rootwrap stuff and policy.json20:52
-openstackstatus- NOTICE: The openstackgerrit IRC bot (gerritbot) will be offline for a short period while we redeploy it on a new server20:52
TheJuliaso this may be a bit extreme, but can you correspond the discovery getting handled by the ironic-inspector your running all the way to the ironic logs?20:58
trandleswhat am I looking for specifically? Just log messages for the enrollment in the ironic logs?21:00
trandlesrunning now21:04
trandlesTheJulia, thanks for the log suggestion. I think I found the error and where "agent_ipmitool" came from...I grabbed an example introspection rule and it had the bad driver (agent_ipmitool). I'm looking for the documentation page where I found the rule I used to see if I inadvertently copied from an old release or if the currrent documentation needs updating.21:18
* trandles bangs head on desk21:25
trandlesit wasn't an old inspector lurking on my system, it was an old database entry21:26
*** zbitter is now known as zaneb21:27
*** zzzeek has joined #openstack-ironic21:38
TheJuliaugh :(21:41
TheJuliaWell, I'm glad you found it, I was getting a bit worried21:41
trandlesGood news is the current documentation is fine for the introspection rule. I had multiple database entries trying to do the same thing and the old one was getting picked up.21:43
TheJuliayeouch but kind of understandable21:46
*** ociuhandu_ has quit IRC22:05
trandlesthanks for your help and patience today TheJulia and iurygregory ... I greatly appreciate it22:16
TheJuliatrandles: no worries, glad you found the issue22:36
*** trandles has quit IRC22:40
*** gregwork has joined #openstack-ironic22:45
*** rcernin has joined #openstack-ironic22:45
jandersgood morning Ironic o/22:52
jandersjumping on a call, back in an hour22:52
*** tkajinam has joined #openstack-ironic22:54
JamesBen_TheJulia: Working on getting our IPMI website public (from the ironic conf recently), trying to migrate it to docker for everyone ;-)22:55
*** trandles_ has joined #openstack-ironic22:59
*** trandles_ is now known as trandles22:59
*** JamesBen_ has quit IRC23:01
*** JamesBenson has joined #openstack-ironic23:04
*** logan- has quit IRC23:05
*** JamesBenson has quit IRC23:10
*** trandles has left #openstack-ironic23:48
*** rloo has quit IRC23:54

Generated by irclog2html.py 2.17.2 by Marius Gedminas - find it at https://mg.pov.lt/irclog2html/!