| janders | back | 00:08 |
|---|---|---|
| *** JamesBenson has joined #openstack-ironic | 01:39 | |
| *** JamesBenson has quit IRC | 01:44 | |
| *** gyee has quit IRC | 01:49 | |
| *** JamesBenson has joined #openstack-ironic | 02:51 | |
| *** JamesBenson has quit IRC | 02:56 | |
| *** mkrai has joined #openstack-ironic | 02:58 | |
| *** rcernin has quit IRC | 03:10 | |
| *** tzumainn has quit IRC | 03:14 | |
| *** stevebaker has quit IRC | 03:15 | |
| *** rcernin has joined #openstack-ironic | 03:17 | |
| *** stevebaker has joined #openstack-ironic | 03:48 | |
| *** ociuhandu has joined #openstack-ironic | 04:07 | |
| *** ociuhandu has quit IRC | 04:12 | |
| *** logan- has joined #openstack-ironic | 04:40 | |
| *** mkrai has quit IRC | 04:44 | |
| *** mkrai has joined #openstack-ironic | 04:46 | |
| *** stevebaker has quit IRC | 04:51 | |
| *** JamesBenson has joined #openstack-ironic | 04:53 | |
| *** JamesBenson has quit IRC | 04:57 | |
| *** abdysn has joined #openstack-ironic | 05:18 | |
| *** JamesBenson has joined #openstack-ironic | 05:36 | |
| *** JamesBenson has quit IRC | 05:40 | |
| *** ricolin has quit IRC | 06:08 | |
| *** weshay|pto has quit IRC | 06:12 | |
| *** weshay_ has joined #openstack-ironic | 06:13 | |
| *** akahat has quit IRC | 06:13 | |
| *** akahat has joined #openstack-ironic | 06:15 | |
| iurygregory | good morning Ironic! | 07:00 |
| janders | good morning iurygregory | 07:02 |
| iurygregory | janders, o/ | 07:03 |
| *** rcernin has quit IRC | 07:07 | |
| *** abdysn has quit IRC | 07:11 | |
| *** abdysn has joined #openstack-ironic | 07:11 | |
| *** ntt_ has joined #openstack-ironic | 07:14 | |
| janders | iurygregory I made some progress with clean_steps work based on our yesterday's chat - thanks! | 07:16 |
| janders | I've got a question though. Testing/debugging the new patch I'm hitting a lot of status codes 400 on the IPA | 07:16 |
| janders | (checking the server side now) | 07:16 |
| iurygregory | awesome! | 07:17 |
| iurygregory | do you have some logs we can look at? | 07:17 |
| *** mkrai has quit IRC | 07:17 | |
| janders | http://paste.openstack.org/show/796752/ | 07:18 |
| janders | this is the server side | 07:18 |
| janders | when the IPA gets 400s, the server logs these "suspicious activity" messages | 07:18 |
| iurygregory | agent_token? | 07:18 |
| janders | it seems to be happening more and more. I used to get it 1-2 times a day and now I don't seem to be able to test cause I hit it most of the time | 07:19 |
| iurygregory | yeah | 07:19 |
| iurygregory | it's agent_token hehe | 07:19 |
| janders | how do I fix it? | 07:19 |
| janders | clear some token DB table like in the good old keystone days? :) | 07:19 |
| iurygregory | I thought we fixed the problems we had with agent_token on deployments | 07:22 |
| janders | perhaps my codebase is a little old | 07:22 |
| iurygregory | when this is happening, like the workflow you did | 07:22 |
| janders | I try clean, which either fails or gets stuck and I abort it | 07:22 |
| janders | and then I do: | 07:22 |
| janders | baremetal node maintenance unset testvm2; baremetal node manage testvm2; baremetal node provide testvm2 | 07:22 |
| janders | which triggers another clean | 07:23 |
| iurygregory | oh ok | 07:23 |
| janders | it's possible that the flow is interrupted due to fault in my patch (it's still WIP) and the token doesn't get wiped... is there an easy-enough way to wipe it manually? | 07:24 |
| iurygregory | yeah | 07:24 |
| iurygregory | let me find how you remove | 07:24 |
| iurygregory | 1min | 07:24 |
| janders | that would be great, thanks iurygregory | 07:24 |
| iurygregory | janders, check your node driver_internal_info | 07:26 |
| janders | http://paste.openstack.org/show/796753/ | 07:27 |
| *** hjensas has quit IRC | 07:27 | |
| janders | http://paste.openstack.org/raw/796753/ might be easier | 07:27 |
| iurygregory | agent_secret_token =D | 07:28 |
| iurygregory | In theory if you remove and re run cleaning it should work | 07:28 |
| iurygregory | in our code we do have some places where we remove since it failed | 07:28 |
| janders | how do I do it? look up --unset operation in the CLI? | 07:30 |
| iurygregory | that's what I'm thinking... | 07:30 |
| iurygregory | I'm not 100% sure we have the support in CLI for this | 07:30 |
| iurygregory | maybe unset would do the work | 07:31 |
| *** yolanda has quit IRC | 07:32 | |
| iurygregory | well we do have [--driver-info <key>] | 07:32 |
| iurygregory | openstack baremetal node unset [--driver-info <key>] <node> | 07:32 |
| *** yolanda has joined #openstack-ironic | 07:33 | |
| iurygregory | janders, I think this would do the job ^ | 07:33 |
| *** Qianbiao has joined #openstack-ironic | 07:33 | |
| *** JamesBenson has joined #openstack-ironic | 07:37 | |
| *** tosky has joined #openstack-ironic | 07:41 | |
| *** JamesBenson has quit IRC | 07:41 | |
| *** dougsz has joined #openstack-ironic | 07:45 | |
| janders | great, thanks! :) will test this shortly | 07:47 |
| rpittau | good morning ironic! o/ | 07:49 |
| *** Lucas_Gray has joined #openstack-ironic | 07:51 | |
| iurygregory | good morning rpittau o/ | 07:52 |
| rpittau | hey iurygregory :) | 07:52 |
| *** mkrai has joined #openstack-ironic | 07:52 | |
| janders | good morning rpittau o/ | 07:56 |
| rpittau | hey janders :) | 07:56 |
| rpittau | janders: it might be you're looking for this https://review.opendev.org/744655 | 07:57 |
| patchbot | patch 744655 - ironic - Wipe agent token and URL on rescue and unrescue (MERGED) - 1 patch set | 07:57 |
| rpittau | mmm that's for rescuing, bad me | 07:58 |
| rpittau | I'm just not sure we're supposed to remove the token manually | 07:59 |
| iurygregory | me too | 08:00 |
| iurygregory | but since janders is doing some testing and things may not complete the workflow that was the only way I saw atm | 08:00 |
| rpittau | so cleaning doesn't complete ? | 08:01 |
| iurygregory | `I try clean, which either fails or gets stuck and I abort it` | 08:01 |
| iurygregory | in his words, yes =) | 08:01 |
| rpittau | ok | 08:01 |
| iurygregory | if it fails we should do the job, if it get stuck and you abort == Big problems | 08:02 |
| gudrutis0 | Hi all | 08:13 |
| *** JamesBenson has joined #openstack-ironic | 08:14 | |
| *** lucasagomes has joined #openstack-ironic | 08:14 | |
| *** vmud213 has joined #openstack-ironic | 08:17 | |
| *** JamesBenson has quit IRC | 08:19 | |
| *** ntt_ has quit IRC | 08:19 | |
| *** tkajinam has quit IRC | 08:19 | |
| *** Abdallahyas has joined #openstack-ironic | 08:23 | |
| janders | sorry guys, was in another chat | 08:24 |
| janders | I did have a couple of cases of it getting stuck | 08:24 |
| janders | (and aborted) | 08:24 |
| janders | sounds like "there be dragons". | 08:25 |
| janders | how big and how scary are the dragons? | 08:25 |
| iurygregory | they are very big | 08:25 |
| iurygregory | and there are a lot of different types | 08:26 |
| iurygregory | Ice, Dark, Fire, Wind, Eletric and so on | 08:26 |
| janders | ok... next time I get my crocodile bite medication from the local pharmacy, I will check if it's any good for dragon bites, too. | 08:26 |
| iurygregory | rpittau, the bifrost release should be 8.3 I assume? =) | 08:26 |
| iurygregory | janders, ++ | 08:27 |
| rpittau | janders: beware of ninja dragons from space | 08:27 |
| rpittau | iurygregory: yeah, lot of features | 08:28 |
| iurygregory | yep =) | 08:28 |
| rpittau | wow, LOT of features | 08:28 |
| janders | am I doing something stupid, or did I manage to break it real bad: | 08:31 |
| janders | http://paste.openstack.org/raw/796754/ | 08:31 |
| rpittau | any core around with a minute to spare? We need https://review.opendev.org/745695 to unlock inspector CI :/ | 08:31 |
| patchbot | patch 745695 - ironic-inspector - Fix requirements job - 1 patch set | 08:31 |
| iurygregory | ++ | 08:33 |
| iurygregory | janders, didn't work =( | 08:34 |
| rpittau | janders: that is what I was afraid of, I think you can't remove the agent token manually | 08:34 |
| iurygregory | sorry >.< | 08:34 |
| janders | time to bring out the SQL client? | 08:35 |
| iurygregory | well SQL would probably work LOL | 08:35 |
| rpittau | janders: can't you trigger a manual clean on that node ? | 08:35 |
| janders | I can but last time I tried it gets 400s on the IPA and the conductor complains about "suspicious activity" | 08:35 |
| iurygregory | because of the invalid token >.< | 08:36 |
| rpittau | yeah\ | 08:36 |
| rpittau | fantastic | 08:36 |
| rpittau | janders: are you using fast-track ? | 08:37 |
| iurygregory | well either SQL | 08:37 |
| iurygregory | or bye bye node | 08:38 |
| janders | I tried forcing power-off between test runs | 08:38 |
| janders | it seemed to have helped a little till recently | 08:38 |
| janders | this time that didn't help either | 08:38 |
| janders | but yeah when it works I do use fast track | 08:38 |
| *** Lucas_Gray has quit IRC | 08:38 | |
| janders | (as in ironic does that by default) | 08:38 |
| *** ntt_ has joined #openstack-ironic | 08:39 | |
| *** Lucas_Gray has joined #openstack-ironic | 08:42 | |
| rpittau | I wonder if you could use rescue in this case, with the patch I posted before | 08:43 |
| rpittau | or I guess the alternative is deleting the node, I'm not sure going through the database would work either | 08:43 |
| rpittau | have you tried restarting the conductor service ? | 08:43 |
| janders | I am not sure - I might have due to code changes, but I will try again | 08:43 |
| * janders is re-testing after conductor restart | 08:45 | |
| iurygregory | rpittau, is there a way to say I want the branch bugfix/8.3 or I need to manually change after running the new-release command? | 08:46 |
| *** hjensas has joined #openstack-ironic | 08:48 | |
| janders | rpittau conductor restart didn't help (and I forced node power-off prior to retest to kill fasttrack too) | 08:48 |
| janders | I will try the SQL apprach | 08:49 |
| janders | s/apprach/approach | 08:49 |
| *** ociuhandu has joined #openstack-ironic | 08:51 | |
| janders | Hum. I was about to do the update after the last failed test. Looked into DB to get the query right and the token was gone | 08:53 |
| janders | is it possible that by using this CLI command: | 08:53 |
| janders | baremetal node maintenance unset testvm2; baremetal node manage testvm2; baremetal node provide testvm2 | 08:53 |
| rpittau | iurygregory: I'm checking, I did that once and I think it was a mix between manual and automated | 08:53 |
| janders | I make things happen to quick one after another, contributing to the problem? | 08:53 |
| iurygregory | rpittau, same | 08:54 |
| rpittau | janders: that should still happen in sequence, so I would say no, but it would help waiting for a node to be in certain state before issuing the next command | 08:55 |
| janders | noted! | 08:55 |
| janders | thanks rpittau | 08:55 |
| *** ociuhandu has quit IRC | 08:56 | |
| janders | now it's working again.. I will keep an eye and try to identify any patterns | 08:57 |
| iurygregory | \o/ | 09:03 |
| *** mkowalski has quit IRC | 09:05 | |
| janders | interesting pattern: | 09:07 |
| janders | if a node gets stuck in cleaning and cleaning is aborted in the stuck state it leaves a mess | 09:07 |
| janders | (token and what not) | 09:07 |
| janders | however | 09:07 |
| janders | if a node is in cleaning but not stuck and then cleaning is aborted, it does clean up | 09:07 |
| janders | so (for now I only tested this once) the recovery method from the busted state might be re-trying cleaning and immediately aborting it and then starting again | 09:08 |
| janders | I hope to fix my code and not test it again but in case I failed I will confirm or reject this hypothesis | 09:08 |
| janders | :) | 09:09 |
| *** mkowalski has joined #openstack-ironic | 09:14 | |
| *** hjensas has quit IRC | 09:19 | |
| *** ociuhandu has joined #openstack-ironic | 09:28 | |
| iurygregory | yay inspector will be fixed \o/ | 09:31 |
| *** yolanda has quit IRC | 09:34 | |
| *** yolanda has joined #openstack-ironic | 09:38 | |
| *** yolanda has quit IRC | 09:38 | |
| *** yolanda has joined #openstack-ironic | 09:38 | |
| rpittau | iurygregory: I left a message on the release channel for the bugfix stuff, in the meantime I saw you worked around that :) | 09:51 |
| *** hjensas has joined #openstack-ironic | 09:51 | |
| *** mkrai has quit IRC | 09:59 | |
| rpittau | we don't install diskimage-bulder from git :/ | 10:02 |
| *** mkrai has joined #openstack-ironic | 10:05 | |
| *** JamesBenson has joined #openstack-ironic | 10:15 | |
| *** Lucas_Gray has quit IRC | 10:16 | |
| *** Qianbiao has quit IRC | 10:19 | |
| *** Lucas_Gray has joined #openstack-ironic | 10:21 | |
| *** JamesBenson has quit IRC | 10:22 | |
| *** mkrai has quit IRC | 10:27 | |
| *** mkrai has joined #openstack-ironic | 10:30 | |
| *** Lucas_Gray has quit IRC | 10:34 | |
| *** ociuhandu has quit IRC | 10:38 | |
| *** ociuhandu has joined #openstack-ironic | 10:39 | |
| *** yolanda has quit IRC | 10:44 | |
| *** Lucas_Gray has joined #openstack-ironic | 10:45 | |
| *** yolanda has joined #openstack-ironic | 10:46 | |
| *** mkrai has quit IRC | 10:49 | |
| *** sshnaidm is now known as sshnaidm|afk | 10:57 | |
| *** ociuhandu_ has joined #openstack-ironic | 10:59 | |
| *** ociuhandu has quit IRC | 11:02 | |
| *** yolanda has quit IRC | 11:03 | |
| *** ociuhandu_ has quit IRC | 11:03 | |
| *** yolanda has joined #openstack-ironic | 11:06 | |
| *** Qianbiao has joined #openstack-ironic | 11:09 | |
| *** Qianbiao has quit IRC | 11:10 | |
| *** Qianbiao has joined #openstack-ironic | 11:10 | |
| *** yolanda has quit IRC | 11:12 | |
| *** k_mouza has joined #openstack-ironic | 11:12 | |
| vmud213 | Hello Ironic! | 11:16 |
| vmud213 | any idea on the network_data property of the node? | 11:16 |
| vmud213 | i am giving it a try with glean installed on the IPA | 11:16 |
| vmud213 | and expecting the glean to disable the DHCP and apply the static configuration provided through "network_data". | 11:17 |
| vmud213 | But apparently that's not happening. | 11:17 |
| vmud213 | Wondering if this was ever tested and tried out? | 11:18 |
| *** ociuhandu has joined #openstack-ironic | 11:18 | |
| *** ociuhandu has quit IRC | 11:23 | |
| *** yolanda has joined #openstack-ironic | 11:25 | |
| *** dougsz has quit IRC | 11:28 | |
| iurygregory | vmud213, hey o/ what elements did you include in the image you are testing? | 11:29 |
| iurygregory | Its on my list to do some tests with dhcp-less deployment | 11:29 |
| vmud213 | ironic-python-agent-builder -o pro-vinay-ssl-glean-ubuntu -e devuser -e selinux-permissive -e simple-init ubuntu | 11:29 |
| vmud213 | Hey iurygregory, Thanks for the response. | 11:30 |
| vmud213 | ANything i am missing? | 11:30 |
| iurygregory | simple-init should do the job, according to the people from diskimage-builder | 11:31 |
| iurygregory | but I do remember TheJulia said something about that it may not complete work and we would need an element like simple-init but that would check more locations... | 11:31 |
| iurygregory | so seems we will need the new element | 11:31 |
| iurygregory | do you have some logs to share so we can look at? | 11:32 |
| vmud213 | I didn't understand why we need a new element | 11:32 |
| iurygregory | let me see if I can find the log from irc 2min | 11:32 |
| vmud213 | as the purpose of the simple-init to install and use Glean which again is the simpler interface for netwrok management | 11:32 |
| vmud213 | Exactly.I am trying to find out them.. WANy idea where can i find? By the time i login to IPA the heartbeat is done and the deploy goews through with the dhcp. | 11:33 |
| iurygregory | vmud213, http://eavesdrop.openstack.org/irclogs/%23openstack-ironic/%23openstack-ironic.2020-08-04.log.html#t2020-08-04T13:27:35 | 11:35 |
| iurygregory | normally /var/log would have something | 11:36 |
| vmud213 | will look into them | 11:36 |
| vmud213 | hmm.. i searched for files "*glean*" but of no use | 11:36 |
| iurygregory | you using a specific deployment tool for your setup? | 11:36 |
| vmud213 | so wondering if the service really kicked off. | 11:36 |
| vmud213 | ofcourse in /var/log . | 11:37 |
| vmud213 | one question.. | 11:38 |
| iurygregory | https://docs.openstack.org/ironic-python-agent/latest/admin/troubleshooting | 11:38 |
| iurygregory | maybe it would help =) | 11:38 |
| iurygregory | sure | 11:38 |
| vmud213 | if an ISO is created with a label "config-2", when the virtual media boots up some process/service should mount the ISO that makes it to appear in in /dev/disk/by-label | 11:40 |
| iurygregory | you mean you create the iso? | 11:42 |
| vmud213 | who does it? Glean itself? | 11:43 |
| vmud213 | If we use redfish virtual meadia boot, the conductor creates the "config-2" labeled ISO and includes that in the virtual media ISO that includes IPA as well | 11:45 |
| *** JamesBenson has joined #openstack-ironic | 11:45 | |
| *** Qianbiao has quit IRC | 11:45 | |
| iurygregory | ironic will generate the iso for vmedia | 11:46 |
| *** JamesBenson has quit IRC | 11:46 | |
| iurygregory | glean is just responsible to do the network config | 11:46 |
| *** JamesBenson has joined #openstack-ironic | 11:46 | |
| vmud213 | Then when the virtualmedia ISO boots up, the "config-2" labeled "vfat" image or "iso" image needs to be first mounted before Glean service can see it . | 11:46 |
| vmud213 | What i don't understand is who mounts the device | 11:47 |
| vmud213 | iurygrgory, the missing part that i am looking for is, before Glean can read the network config it should find the "config-2" labeld device in the system using using "blkid" | 11:49 |
| iurygregory | ipa is responsible for mounting | 11:50 |
| vmud213 | Now blkid can detect the labeled device unless someone mounts the devices | 11:50 |
| vmud213 | oh...ok.. | 11:50 |
| vmud213 | so glean should run only after IPA service. | 11:50 |
| vmud213 | Is my understanding clear? | 11:50 |
| vmud213 | But wait. IPA does it for user image. Not in the context of deploy image. | 11:51 |
| *** k_mouza has quit IRC | 11:52 | |
| vmud213 | this network info is consumed by deploy image and not user image i suppose. That's the whole point of this feature i suppose. | 11:52 |
| vmud213 | Am i missing anything? | 11:52 |
| iurygregory | so, the workflow fo this would be something like | 11:54 |
| iurygregory | you have a deploy ramdisk with the network configuration for the network_data field | 11:55 |
| vmud213 | ok | 11:55 |
| iurygregory | you need to pass a .json file with the correct format etc since Glean will consume | 11:55 |
| iurygregory | Ironic conductor validates the .json | 11:55 |
| janders | see you tomorrow Ironic o/ | 11:56 |
| iurygregory | ironic builds a new config drive image and places the json file as content for network_data field | 11:56 |
| janders | are you aware of any good examples how to use bifrost developer mode? | 11:57 |
| iurygregory | Ironic build the bootable iso | 11:57 |
| janders | I had a look at the code but wasn't sure how to get it to grab one custom source directory (which would be my patch) and install the rest as usual | 11:57 |
| vmud213 | ok | 11:58 |
| *** abdysn has quit IRC | 11:58 | |
| *** Abdallahyas has quit IRC | 11:58 | |
| vmud213 | the bootable ISO includes a file with iso/vfat format with "config-2" label that's placed in the root directory of the ISO | 11:59 |
| *** abdysn has joined #openstack-ironic | 11:59 | |
| iurygregory | janders, I haven't use myself the feature but I will look to see if I can provide something for you | 11:59 |
| iurygregory | vmud213, correct | 11:59 |
| vmud213 | Now, when ISO boots up, assuming no DHCP glean kicks off and tries to configure the network. THe IPA is not in picture yet | 12:00 |
| vmud213 | now for Glean to identify the "config-2" block device either glean or some other service would have already mounted the ISO present in the root directory | 12:02 |
| vmud213 | Now who does it? That's where i am not clear :) | 12:02 |
| *** yolanda has quit IRC | 12:03 | |
| vmud213 | But anyway, i am wasting your time.. | 12:03 |
| vmud213 | Don't really mean to. | 12:03 |
| iurygregory | Glean runs inside the ramdisk | 12:04 |
| iurygregory | ipa takes care of the ramdisk | 12:04 |
| vmud213 | ok. | 12:04 |
| iurygregory | rpittau, what I'm saying is correct? ( I assume it's) | 12:04 |
| iurygregory | vmud213, you are not wasting my time =) | 12:05 |
| vmud213 | iurygregory, from what you referred in the etherpad, it should work. | 12:07 |
| vmud213 | correct. | 12:07 |
| vmud213 | that's a good news. will try to see what i am missing. | 12:08 |
| janders | iurygregory only if you have time - if not, I will look tomorrow. Thanks heaps! :) | 12:09 |
| iurygregory | vmud213, awesome let us know the result | 12:09 |
| vmud213 | Hey | 12:09 |
| * iurygregory hopes simple-init will work | 12:09 | |
| iurygregory | janders, ack o/ | 12:09 |
| vmud213 | i missed one statement from Julia | 12:09 |
| vmud213 | "at a minimum we likely need a loop attach of our iso for it to be found" | 12:10 |
| vmud213 | As per TheJulia's comment @13.29 | 12:11 |
| iurygregory | yeah | 12:11 |
| vmud213 | Thanks iurygregory. Have a good day ! | 12:15 |
| iurygregory | maybe we will need to change something on ipa I'm thinking now, I will look at the code to see how it would be | 12:15 |
| *** yolanda has joined #openstack-ironic | 12:16 | |
| *** Lucas_Gray has quit IRC | 12:17 | |
| *** ntt__ has joined #openstack-ironic | 12:23 | |
| *** ntt_ has quit IRC | 12:23 | |
| *** dougsz has joined #openstack-ironic | 12:24 | |
| *** vmud213 has quit IRC | 12:29 | |
| rpittau | iurygregory: that was correct | 12:34 |
| rpittau | bye janders o/ | 12:35 |
| *** ianychoi has quit IRC | 12:40 | |
| *** Lucas_Gray has joined #openstack-ironic | 12:51 | |
| *** ociuhandu has joined #openstack-ironic | 12:56 | |
| *** ntt_ has joined #openstack-ironic | 12:56 | |
| *** ntt__ has quit IRC | 12:59 | |
| iurygregory | I'm wondering if we need to add some bits to ipa regarding the specific lable that glean searchs for | 13:03 |
| *** yolanda has quit IRC | 13:07 | |
| *** Qianbiao has joined #openstack-ironic | 13:08 | |
| *** rloo has joined #openstack-ironic | 13:09 | |
| rpittau | wouldn't hurt :) | 13:09 |
| *** tzumainn has joined #openstack-ironic | 13:11 | |
| iurygregory | I'm trying to figure out where XD | 13:18 |
| *** ricolin has joined #openstack-ironic | 13:22 | |
| *** sshnaidm|afk is now known as sshnaidm | 13:38 | |
| * TheJulia blinks | 13:42 | |
| * TheJulia thinks lots of coffee may be needed | 13:42 | |
| rpittau | good morning TheJulia :) | 13:43 |
| *** ianychoi has joined #openstack-ironic | 13:44 | |
| *** trandles has joined #openstack-ironic | 13:46 | |
| iurygregory | good morning TheJulia =) | 13:47 |
| iurygregory | just to more days for the long weekend \o/ | 13:47 |
| TheJulia | except I'm driving like 1500 miles this weekend | 13:49 |
| rpittau | oO | 13:49 |
| iurygregory | WOW | 13:49 |
| TheJulia | Now to boggle your mind... I've done that in a motorhome once which is ~2x times impacting on the human body | 13:51 |
| *** yolanda has joined #openstack-ironic | 13:51 | |
| trandles | Another long road trip TheJulia...coming through NM again? | 13:51 |
| TheJulia | trandles: yeah, a very quick visit | 13:52 |
| iurygregory | TheJulia, omg how many days you take to travel the 1500 miles? | 13:52 |
| TheJulia | just two | 13:52 |
| iurygregory | I was thinking 3 or 4 | 13:53 |
| iurygregory | O.o | 13:53 |
| trandles | Be safe! | 13:53 |
| TheJulia | trandles: will do! | 13:53 |
| TheJulia | iurygregory: I mean, it might take me 3 but I don't anticipiate it. I should just be able to melt into the car and go | 13:54 |
| iurygregory | good luck =) | 13:54 |
| rpittau | hope you'll be able to enjoy the travel, despite the melting :) | 13:55 |
| TheJulia | Heh, I was thinking of taking my camera bag with me | 13:57 |
| *** abdysn has quit IRC | 14:06 | |
| rpittau | mock still surprises me sometimes, to say that politely | 14:08 |
| TheJulia | good way, bad way? | 14:10 |
| iurygregory | I would say bad since he said "politely" | 14:14 |
| rpittau | very bad | 14:31 |
| rpittau | well, my change in dib to fix focal ci is failing in all other jobs, looks like a victory | 14:35 |
| rpittau | TheJulia: iwhen you have a minute, a couple of backports for ipa https://review.opendev.org/744863 https://review.opendev.org/745493 | 14:41 |
| patchbot | patch 744863 - ironic-python-agent (stable/ussuri) - Fix bootloader install issue with MDRAID - 1 patch set | 14:41 |
| patchbot | patch 745493 - ironic-python-agent (bugfix/6.2) - Ignore devices with size 0 when collecting inventory - 1 patch set | 14:41 |
| *** priteau has joined #openstack-ironic | 14:59 | |
| iurygregory | gerritbot is on PTO | 15:00 |
| iurygregory | 0 | 15:02 |
| iurygregory | rpittau, I've pushed the releases for inspector and bifrost, do you need help with any other? | 15:12 |
| *** vmud213 has joined #openstack-ironic | 15:13 | |
| rpittau | iurygregory: thanks! I should be ok with ironic and ironic-python-agent | 15:14 |
| iurygregory | cool =) | 15:14 |
| * iurygregory back fighting privsep and tempest tests | 15:14 | |
| vmud213 | Can anyone take a look at the following patches | 15:16 |
| vmud213 | https://review.opendev.org/#/c/739174/ | 15:16 |
| patchbot | patch 739174 - ironic - Decouple the ISO creation logic from redfish - 7 patch sets | 15:16 |
| vmud213 | https://review.opendev.org/#/c/742936/ | 15:16 |
| patchbot | patch 742936 - ironic - Allow HttpImageService to accept custom certificate - 5 patch sets | 15:16 |
| vmud213 | dtantsur, TheJulia would you mind reviewing them if time permits ? | 15:17 |
| *** rloo has quit IRC | 15:17 | |
| vmud213 | 742936 is reviewed once and i addressed the comments | 15:17 |
| *** rloo has joined #openstack-ironic | 15:18 | |
| TheJulia | vmud213: I should be able to look later today, seems like today is going to be code and code reviews :) | 15:18 |
| vmud213 | Thanks :) | 15:18 |
| *** Qianbiao has quit IRC | 15:26 | |
| *** Qianbiao has joined #openstack-ironic | 15:27 | |
| *** rloo has quit IRC | 15:30 | |
| *** rloo has joined #openstack-ironic | 15:31 | |
| rpittau | iurygregory: maybe too late, but if you want to test it -> https://review.opendev.org/745918 | 15:34 |
| patchbot | patch 745918 - releases - Add intermediate-branch arg to new-release script - 1 patch set | 15:34 |
| rpittau | ipa is in the release queue, I will leave ironic for tomorrow as hopefully a couple of patches will be merged | 15:42 |
| iurygregory | rpittau, well I can test locally lol =D | 15:47 |
| rpittau | :) | 15:47 |
| iurygregory | rpittau, ++ =) | 15:48 |
| rpittau | iurygregory: thanks :) | 15:49 |
| iurygregory | works like a charm! | 15:49 |
| rpittau | cool, let's see if the elders of the internet like it :) | 15:49 |
| iurygregory | yeah | 15:49 |
| *** rloo has quit IRC | 15:51 | |
| iurygregory | ERROR: No matching distribution found for lxml===4.5.2 WUT | 15:53 |
| rpittau | zuul was magnanimous :) | 15:53 |
| *** rloo has joined #openstack-ironic | 15:53 | |
| rpittau | iurygregory: where? what? how? | 15:54 |
| iurygregory | ironic-lib | 15:54 |
| iurygregory | testing privsep | 15:54 |
| iurygregory | only on the functional jobs | 15:54 |
| iurygregory | requirements job and l-c jobs are green LOL | 15:54 |
| iurygregory | magic | 15:55 |
| *** ntt_ has quit IRC | 15:55 | |
| *** lucasagomes has quit IRC | 15:57 | |
| rpittau | iurygregory: which patch ? | 15:57 |
| iurygregory | rpittau, 745536 I'm looking at the logs on https://zuul.openstack.org/status | 15:58 |
| iurygregory | since two jobs are still running | 15:58 |
| iurygregory | the other job that failed ERROR: No matching distribution found for PyMySQL===0.10.0 | 15:58 |
| * iurygregory goes to infra | 15:58 | |
| rpittau | might be vexxhost still acting up | 15:59 |
| *** Qianbiao has quit IRC | 15:59 | |
| iurygregory | yeah | 16:00 |
| iurygregory | I sent a message to my friend that works at vexxhost hehe | 16:01 |
| rpittau | sounds menacing | 16:01 |
| iurygregory | nah | 16:02 |
| rpittau | :) | 16:02 |
| iurygregory | we studied and worked on the same university in Brazil | 16:02 |
| * rpittau needs ice-cream | 16:03 | |
| rpittau | ttyl o/ | 16:03 |
| iurygregory | bye o/ | 16:03 |
| iurygregory | yay at least one job failed but is a valid failure | 16:05 |
| iurygregory | PermissionError: [Errno 1] Operation not permitted: '/var/lib/ironic/master_images/tmp1b94pp_z/c56a2e9e-77d8-4a63-83d6-c245933382dd' -> '/var/lib/ironic/master_images/c56a2e9e-77d8-4a63-83d6-c245933382dd.converted' \o/ | 16:06 |
| iurygregory | at least it gives me a point | 16:06 |
| *** mkrai has joined #openstack-ironic | 16:08 | |
| *** vmud213 has quit IRC | 16:28 | |
| *** dougsz has quit IRC | 16:33 | |
| *** Lucas_Gray has quit IRC | 16:36 | |
| *** trandles has quit IRC | 16:45 | |
| *** mgoddard has quit IRC | 16:56 | |
| *** mgoddard has joined #openstack-ironic | 17:02 | |
| *** mkrai has quit IRC | 17:09 | |
| *** trandles has joined #openstack-ironic | 17:11 | |
| *** priteau has quit IRC | 17:27 | |
| *** mgoddard has quit IRC | 17:28 | |
| *** mgoddard has joined #openstack-ironic | 17:34 | |
| *** dking has joined #openstack-ironic | 17:43 | |
| dking | I don't think that this is specifically an Ironic thing, but has anybody here experienced an issue where a certain type of server experiences high power consumption/temperature after deploying images, which remains high until boot, even though CPU usage and I/O seem to be minimal? | 17:46 |
| dking | We have several types of server to which we can deploy without any issue, but for one type of server, that behavior is consistent. I noticed that if I use a DIB image without growroot, it seems to be fine, but if I either use growroot, or attempt to format the image with qemu, then deploying that image causes the issue on just that type of server. | 17:49 |
| *** mgoddard has quit IRC | 17:55 | |
| *** gyee has joined #openstack-ironic | 17:56 | |
| *** mgoddard has joined #openstack-ironic | 18:02 | |
| * TheJulia re-reads to make sure she is groking this properly | 18:02 | |
| TheJulia | dking: I've never heard of such a thing but it kind of makes me wonder if maybe a disk controller is doing a background scrub? | 18:02 |
| *** vdrok_ has joined #openstack-ironic | 18:13 | |
| *** vkmc_ has joined #openstack-ironic | 18:13 | |
| *** vdrok has quit IRC | 18:20 | |
| *** vkmc has quit IRC | 18:20 | |
| *** vkmc_ is now known as vkmc | 18:20 | |
| *** vdrok_ is now known as vdrok | 18:20 | |
| *** mkowalski has quit IRC | 18:21 | |
| *** mgoddard has quit IRC | 18:23 | |
| *** ricolin has quit IRC | 18:23 | |
| *** mgoddard has joined #openstack-ironic | 18:25 | |
| *** mkowalski has joined #openstack-ironic | 18:28 | |
| *** rloo has quit IRC | 18:38 | |
| dking | TheJulia: Yeah, it's an odd issue. Do you think that such a thing would happen consistently on first boot? I forgot to mention that the issue typically goes away when the server is rebooted. However, I've heard that the issue occurs again during normal operation, though I haven't been able to verify that myself. | 18:46 |
| dking | I also wonder if there might possibly be some difference in the BIOS settings for these machines. | 18:46 |
| *** Lucas_Gray has joined #openstack-ironic | 18:50 | |
| *** Lucas_Gray has quit IRC | 19:00 | |
| TheJulia | dking: yeah, that does kind of sound like the controller scrubbing | 19:17 |
| TheJulia | which will run at a low io priority, but it will run constantly which can consume power | 19:17 |
| TheJulia | and won't be os visible | 19:17 |
| *** k_mouza has joined #openstack-ironic | 19:23 | |
| iurygregory | TheJulia, fyi the releases for bifrost and inspector https://review.opendev.org/745809 https://review.opendev.org/745915 | 19:26 |
| patchbot | patch 745809 - releases - Release Bifrost 8.3 for Victoria - 1 patch set | 19:26 |
| patchbot | patch 745915 - releases - Release ironic-inspector bugfix/10.3 - 1 patch set | 19:26 |
| *** k_mouza has quit IRC | 19:27 | |
| TheJulia | thanks! | 19:33 |
| TheJulia | iurygregory: lgtm | 19:48 |
| TheJulia | approved | 19:48 |
| iurygregory | TheJulia, tks o/ | 19:49 |
| iurygregory | I've put some rechecks in ironic patches to see if CI helps since rpittau will push the release tomorrow | 19:49 |
| TheJulia | it looks like CI is dead. Again | 19:52 |
| iurygregory | well there are some problems with the vexxhost | 19:53 |
| iurygregory | so we need to keep fingers crossed hehe | 19:53 |
| TheJulia | Yeah, looks connectivity related | 19:55 |
| iurygregory | according to infra some ipv6 problems | 20:01 |
| iurygregory | ok this is weird O.o ironic.conf should have [ironic_lib]root_helper config "oslo_config.cfg.NoSuchOptError: no such option root_helper in group [ironic_lib]" | 20:04 |
| TheJulia | starting in what version? | 20:10 |
| iurygregory | well according to our docs we have https://docs.openstack.org/ironic/latest/configuration/sample-config.html | 20:11 |
| TheJulia | hmmm | 20:12 |
| iurygregory | https://github.com/openstack/ironic-lib/blob/6c9d5dc58b976c30be95340b8799daf485203994/ironic_lib/utils.py#L41 | 20:12 |
| TheJulia | and you get no such option group where? | 20:13 |
| iurygregory | I was trying to initialize privsep on ironic and push the ironic-lib with the Depends-On, because I got PermissionError before | 20:13 |
| TheJulia | so ironic_lib things will always have to merge first | 20:13 |
| TheJulia | because of upper constraints | 20:13 |
| iurygregory | humm so maybe I need to initialize privsep in some place on ironic-lib | 20:14 |
| TheJulia | well you can't test ironic hcanges with unreleased ironiclib chagnes | 20:14 |
| TheJulia | and... depending on how your tyring to leverage the var in ironic, you may actually need to define it | 20:15 |
| iurygregory | it was more test ironic on ironic-lib... | 20:15 |
| TheJulia | oh | 20:16 |
| TheJulia | hmm | 20:16 |
| iurygregory | since the changes only in ironic-lib went to PermissionError when trying to copy the image | 20:17 |
| iurygregory | because the qemu-img convert is on privsep | 20:17 |
| iurygregory | time to call it a day o/ | 20:20 |
| TheJulia | hmm | 20:28 |
| TheJulia | interesting | 20:28 |
| TheJulia | goodnight | 20:28 |
| *** rloo has joined #openstack-ironic | 20:39 | |
| *** rloo has quit IRC | 20:44 | |
| *** rloo has joined #openstack-ironic | 20:44 | |
| *** Lucas_Gray has joined #openstack-ironic | 20:52 | |
| *** rh-jelabarre has quit IRC | 20:52 | |
| *** rh-jelabarre has joined #openstack-ironic | 20:52 | |
| *** stevebaker has joined #openstack-ironic | 21:05 | |
| *** Lucas_Gray has quit IRC | 21:18 | |
| *** rh-jelabarre has quit IRC | 21:23 | |
| stevebaker | morning | 21:32 |
| bfournie | good morning stevebaker | 21:34 |
| TheJulia | good morning stevebaker | 22:17 |
| * TheJulia misses gerritbot | 22:17 | |
| TheJulia | arne_wiebalck: https://review.opendev.org/745972 | 22:17 |
| patchbot | patch 745972 - ironic-inspector - Fix URL parsing to enable redfish_address matching - 1 patch set | 22:17 |
| stevebaker | Can I borrow some of your brain cycles to talk about node locking at some point? | 22:19 |
| TheJulia | stevebaker: will I need whiskey? | 22:20 |
| TheJulia | better question, should I have a glass of whiskey for said discussion | 22:20 |
| stevebaker | TheJulia: I mean, of course. But I don't think its a harrowing topic | 22:21 |
| TheJulia | heh | 22:22 |
| TheJulia | well, It is about the end of my day too, which was partly why I asked | 22:22 |
| stevebaker | TheJulia: let me know when you're charged and ready | 22:23 |
| TheJulia | give me like 10-15 minutes | 22:23 |
| TheJulia | I just need to get words into a bz | 22:23 |
| *** rcernin has joined #openstack-ironic | 22:43 | |
| *** trandles has quit IRC | 22:43 | |
| *** rcernin has quit IRC | 22:43 | |
| *** rcernin has joined #openstack-ironic | 22:43 | |
| *** tkajinam has joined #openstack-ironic | 22:57 | |
| TheJulia | stevebaker: where shall we chat? | 23:06 |
| janders | good moring/afternoon Ironic o/ | 23:08 |
| *** tosky has quit IRC | 23:13 | |
| *** janders has quit IRC | 23:23 | |
| *** openstackgerrit has joined #openstack-ironic | 23:29 | |
| openstackgerrit | Merged openstack/ironic master: Add RPC objects for deployment API https://review.opendev.org/703778 | 23:29 |
| *** janders has joined #openstack-ironic | 23:43 | |
| janders | sorry, router reboot | 23:44 |
Generated by irclog2html.py 2.17.2 by Marius Gedminas - find it at https://mg.pov.lt/irclog2html/!