| *** tosky has quit IRC | 00:30 | |
| *** mgoddard has quit IRC | 00:51 | |
| *** mgoddard has joined #openstack-ironic | 00:57 | |
| *** ayoung has quit IRC | 01:32 | |
| *** tzumainn has quit IRC | 01:45 | |
| openstackgerrit | Iury Gregory Melo Ferreira proposed openstack/ironic-inspector stable/ussuri: Fix docs https://review.opendev.org/c/openstack/ironic-inspector/+/775727 | 01:58 |
|---|---|---|
| *** rcernin has quit IRC | 02:31 | |
| *** k_mouza has joined #openstack-ironic | 02:50 | |
| *** zzzeek has quit IRC | 02:53 | |
| *** k_mouza has quit IRC | 02:54 | |
| *** zzzeek has joined #openstack-ironic | 02:55 | |
| *** rcernin has joined #openstack-ironic | 03:23 | |
| *** rcernin has quit IRC | 03:28 | |
| *** rcernin has joined #openstack-ironic | 03:28 | |
| *** Qianbiao has joined #openstack-ironic | 03:38 | |
| openstackgerrit | Iury Gregory Melo Ferreira proposed openstack/ironic-inspector stable/train: Fix docs https://review.opendev.org/c/openstack/ironic-inspector/+/775732 | 03:43 |
| *** stendulker has joined #openstack-ironic | 05:28 | |
| openstackgerrit | Merged openstack/bifrost master: Restore the conditional verbosity in bash scripts https://review.opendev.org/c/openstack/bifrost/+/775678 | 05:34 |
| *** mkrai has joined #openstack-ironic | 05:35 | |
| *** ianw has quit IRC | 06:18 | |
| *** ianw has joined #openstack-ironic | 06:19 | |
| *** zzzeek has quit IRC | 06:26 | |
| *** zzzeek has joined #openstack-ironic | 06:29 | |
| *** mugsie has quit IRC | 06:37 | |
| *** mkrai has quit IRC | 06:38 | |
| *** mkrai has joined #openstack-ironic | 06:41 | |
| *** k_mouza has joined #openstack-ironic | 06:50 | |
| *** k_mouza has quit IRC | 06:55 | |
| *** moshiur has joined #openstack-ironic | 07:15 | |
| *** mkrai has quit IRC | 07:23 | |
| *** pmannidi_ has quit IRC | 07:37 | |
| arne_wiebalck | Good morning, ironic! | 07:41 |
| *** pmannidi has joined #openstack-ironic | 07:43 | |
| janders | good morning arne_wiebalck o/ | 07:48 |
| arne_wiebalck | hey janders o/ | 07:48 |
| *** rpittau|afk is now known as rpittau | 08:37 | |
| rpittau | good morning ironic! o/ | 08:37 |
| *** rcernin has quit IRC | 08:40 | |
| *** tosky has joined #openstack-ironic | 08:44 | |
| openstackgerrit | Arun S A G proposed openstack/ironic master: Add anaconda configuration and template https://review.opendev.org/c/openstack/ironic/+/772166 | 08:50 |
| janders | good morning rpittau o/ | 08:52 |
| rpittau | hey janders :) | 08:52 |
| *** dtantsur|afk is now known as dtantsur | 08:53 | |
| dtantsur | morning ironic | 08:53 |
| janders | hey dtantsur o/ | 08:53 |
| janders | dtantsur regarding https://review.opendev.org/c/openstack/ironic-python-agent/+/771904/27/ironic_python_agent/hardware.py#1292 would you like me to rename enable_ata_secure_erase to enable_secure_erase and use it to control both SATA and NVMe secure erase? Makes sense, just making sure I understand correctly. | 08:55 |
| dtantsur | janders: hi, well, that's an option. I'm not sure if there is a valid case for enabling e.g. only ATA erase.. | 08:58 |
| dtantsur | (e.g. our hardware has bad ATA controllers, but NVMe is fine..) | 08:58 |
| openstackgerrit | Dmitry Tantsur proposed openstack/bifrost stable/victoria: Restore the conditional verbosity in bash scripts https://review.opendev.org/c/openstack/bifrost/+/775763 | 09:01 |
| janders | dtantsur I was more thinking of conservative operators going "SATA Sec Erase has been around for ages, so that's fine to use, that new NVMe cleaning thing is new so let's leave it off for a release or two" | 09:01 |
| janders | it's not only the code perspective, but the entire NVMe secure erasure being still relatively new to people (partially because of NVMes being new not because of the feature) | 09:02 |
| janders | but I'm happy either way | 09:02 |
| *** dougsz has joined #openstack-ironic | 09:02 | |
| janders | what do you think? | 09:03 |
| *** lucasagomes has joined #openstack-ironic | 09:08 | |
| *** stendulker has quit IRC | 09:13 | |
| *** rcernin has joined #openstack-ironic | 09:18 | |
| openstackgerrit | Riccardo Pittau proposed openstack/ironic-python-agent master: [WIP] Use json for lsblk output https://review.opendev.org/c/openstack/ironic-python-agent/+/775391 | 09:20 |
| rpittau | very quick trivial approval if anyone has a minute pretty please https://review.opendev.org/c/openstack/ironic-inspector/+/773316 | 09:22 |
| rpittau | dtantsur: good morning! :) I fixed the running times for the unit tests in https://review.opendev.org/c/openstack/ironic/+/376574 | 09:24 |
| dtantsur | janders: okay, let's leave two options for now | 09:27 |
| dtantsur | rpittau: nice, thanks! | 09:27 |
| openstackgerrit | Dmitry Tantsur proposed openstack/ironic master: Replace pysendfile with os.sendfile https://review.opendev.org/c/openstack/ironic/+/772066 | 09:33 |
| *** rcernin has quit IRC | 09:35 | |
| rpittau | dtantsur: I was reading https://review.opendev.org/c/openstack/ironic-lib/+/774931 and wondering if it would make sense to just use lsblk json output | 09:47 |
| dtantsur | rpittau: I think blkid is more low-level and outputs stuff that lsblk does not | 09:48 |
| dtantsur | I do remember there was a problem with that. Maybe that lsblk in CentOS 7 does not have --json? | 09:48 |
| dtantsur | or was it Bionic/Xenial/...? | 09:48 |
| rpittau | oh yeah, centos 7 doesn't have the version for that | 09:48 |
| dtantsur | I'm fine with changing, but I'd rather do it gradually, so that we don't have to revert the whole thing | 09:48 |
| dtantsur | well, yeah. CentOS 7 is something we still support for IPA. | 09:49 |
| rpittau | mmm ok I guess my change in IPA will have to wait then | 09:51 |
| dtantsur | unfortunately, yes :( | 09:52 |
| dtantsur | Apparently, CentOS 8 dropped some drivers that people still used | 09:53 |
| *** zzzeek has quit IRC | 09:53 | |
| *** zzzeek has joined #openstack-ironic | 09:53 | |
| *** mkrai has joined #openstack-ironic | 09:55 | |
| openstackgerrit | OpenStack Release Bot proposed openstack/bifrost bugfix/10.1: Update .gitreview for bugfix/10.1 https://review.opendev.org/c/openstack/bifrost/+/775787 | 09:55 |
| dtantsur | yay, releases | 09:55 |
| * dtantsur wonders if we need to run a CI job with centos 7 | 09:59 | |
| rpittau | if we support it, yes :) | 10:00 |
| rpittau | I was wondering the same as not seeing any may be misleading | 10:00 |
| rpittau | I wonder if it's too late to propose a new version of util-linux to centos 7 :D | 10:01 |
| openstackgerrit | Merged openstack/ironic-inspector stable/ussuri: Fix docs https://review.opendev.org/c/openstack/ironic-inspector/+/775727 | 10:11 |
| openstackgerrit | Merged openstack/ironic-inspector stable/train: Fix docs https://review.opendev.org/c/openstack/ironic-inspector/+/775732 | 10:12 |
| openstackgerrit | Merged openstack/ironic-inspector stable/victoria: Avoid a full install in tox environments that do not need it https://review.opendev.org/c/openstack/ironic-inspector/+/775704 | 10:12 |
| *** derekh has joined #openstack-ironic | 10:13 | |
| openstackgerrit | Merged openstack/bifrost stable/victoria: Restore the conditional verbosity in bash scripts https://review.opendev.org/c/openstack/bifrost/+/775763 | 10:14 |
| *** derekh has quit IRC | 10:29 | |
| *** derekh has joined #openstack-ironic | 10:29 | |
| *** ociuhandu has joined #openstack-ironic | 10:31 | |
| *** ociuhandu has quit IRC | 10:40 | |
| openstackgerrit | Kartikeya Jain proposed openstack/ironic-python-agent master: Fixes local boot for partition images https://review.opendev.org/c/openstack/ironic-python-agent/+/775799 | 10:43 |
| *** ociuhandu has joined #openstack-ironic | 10:46 | |
| *** ociuhandu has quit IRC | 10:46 | |
| *** ociuhandu has joined #openstack-ironic | 10:47 | |
| *** ociuhandu has quit IRC | 10:52 | |
| *** k_mouza has joined #openstack-ironic | 11:00 | |
| dtantsur | rpittau: way too late :) | 11:01 |
| dtantsur | I think RHEL 7 is in security-only mode atm | 11:01 |
| *** ociuhandu has joined #openstack-ironic | 11:01 | |
| rpittau | yeah :/ | 11:01 |
| iurygregory | good morning o/ | 11:04 |
| rpittau | hey iurygregory :) | 11:07 |
| iurygregory | o/ | 11:08 |
| *** alexmcleod has joined #openstack-ironic | 11:13 | |
| *** ociuhandu has quit IRC | 11:13 | |
| janders | good morning iurygregory o/ | 11:13 |
| *** mugsie has joined #openstack-ironic | 11:13 | |
| iurygregory | o/ | 11:15 |
| *** mkrai has quit IRC | 11:17 | |
| *** ociuhandu has joined #openstack-ironic | 11:17 | |
| openstackgerrit | OpenStack Release Bot proposed openstack/ironic bugfix/16.2: Update .gitreview for bugfix/16.2 https://review.opendev.org/c/openstack/ironic/+/775806 | 11:18 |
| openstackgerrit | Riccardo Pittau proposed openstack/networking-generic-switch stable/victoria: [DNM] test victoria CI https://review.opendev.org/c/openstack/networking-generic-switch/+/775807 | 11:20 |
| openstackgerrit | Riccardo Pittau proposed openstack/networking-generic-switch stable/ussuri: [DNM] test ussuri CI https://review.opendev.org/c/openstack/networking-generic-switch/+/775808 | 11:21 |
| *** anuradha1904 has joined #openstack-ironic | 11:22 | |
| openstackgerrit | Kartikeya Jain proposed openstack/ironic-python-agent master: Fixes local boot for partition images https://review.opendev.org/c/openstack/ironic-python-agent/+/775799 | 11:30 |
| *** moshiur has quit IRC | 11:39 | |
| iurygregory | dtantsur, tks for aproving the doc fixes in the stable branches o/ I'm still trying to fix stable/train | 11:52 |
| iurygregory | s/aproving/approving | 11:52 |
| iurygregory | lol I pushed the fix for stable/train | 11:55 |
| iurygregory | I was probably sleeping XD | 11:55 |
| *** ociuhandu has quit IRC | 12:13 | |
| *** ociuhandu has joined #openstack-ironic | 12:14 | |
| openstackgerrit | Merged openstack/ironic stable/victoria: Move the IPv6 job to the experimental pipeline https://review.opendev.org/c/openstack/ironic/+/775608 | 12:14 |
| *** tosin has joined #openstack-ironic | 12:18 | |
| *** ociuhandu has quit IRC | 12:21 | |
| openstackgerrit | Merged openstack/python-ironic-inspector-client master: Remove redundant items from CI jobs https://review.opendev.org/c/openstack/python-ironic-inspector-client/+/754692 | 12:35 |
| arne_wiebalck | dtantsur: TheJulia: for the thundering herd issue on startup, I will see if I can get a DB trace to see where all these queries come from: there is maybe sth to optimise in Ironic's queries which could reduce the load to a level where a staggered start or jitter on the periodics won't be necessary ... I may come back to you for advice/input/ideas :) | 12:45 |
| janders | see you tomorrow Ironic o/ | 12:47 |
| janders | was hoping to upload revised NVMe patch today, but got stuck on rewriting unit tests... trying to do that at 11pm is probably not the best idea :) | 12:48 |
| janders | much easier with a fresh mind | 12:48 |
| *** alexmcleod has quit IRC | 12:49 | |
| *** moshiur has joined #openstack-ironic | 12:51 | |
| openstackgerrit | Merged openstack/ironic-inspector stable/train: Fix memcached host address. https://review.opendev.org/c/openstack/ironic-inspector/+/775641 | 12:55 |
| iurygregory | bye janders o/ | 13:05 |
| openstackgerrit | Merged openstack/bifrost bugfix/10.1: Update .gitreview for bugfix/10.1 https://review.opendev.org/c/openstack/bifrost/+/775787 | 13:15 |
| *** alexmcleod has joined #openstack-ironic | 13:16 | |
| *** monica_pardhi has joined #openstack-ironic | 13:23 | |
| *** ociuhandu has joined #openstack-ironic | 13:23 | |
| openstackgerrit | Merged openstack/ironic bugfix/16.2: Update .gitreview for bugfix/16.2 https://review.opendev.org/c/openstack/ironic/+/775806 | 13:26 |
| openstackgerrit | Verification of a change to openstack/ironic-inspector failed: Fix memcached host address. https://review.opendev.org/c/openstack/ironic-inspector/+/775640 | 13:32 |
| *** rcernin has joined #openstack-ironic | 13:33 | |
| *** rcernin has quit IRC | 13:38 | |
| *** k_mouza has quit IRC | 13:39 | |
| *** k_mouza has joined #openstack-ironic | 13:39 | |
| *** monica_pardhi has quit IRC | 13:42 | |
| *** k_mouza_ has joined #openstack-ironic | 13:49 | |
| *** k_mouza__ has joined #openstack-ironic | 13:50 | |
| *** k_mouza_ has quit IRC | 13:50 | |
| *** k_mouza has quit IRC | 13:51 | |
| *** k_mouza__ has quit IRC | 13:53 | |
| *** k_mouza has joined #openstack-ironic | 13:54 | |
| *** uzumaki has joined #openstack-ironic | 13:55 | |
| openstackgerrit | Riccardo Pittau proposed openstack/ironic-python-agent master: Move some raid specific functions to raid_utils https://review.opendev.org/c/openstack/ironic-python-agent/+/774854 | 14:00 |
| TheJulia | good morning | 14:03 |
| dtantsur | good morning TheJulia | 14:05 |
| uzumaki | anybody have an idea how to supress the verbose HTTP call output with openstack-sdk? | 14:05 |
| uzumaki | Morning dtantsur TheJulia o/ | 14:05 |
| dtantsur | please elaborate | 14:05 |
| uzumaki | any api call that I make, results in a verbose output on STDOUT, with a dump of HTTP calls, I want to get rid of it | 14:06 |
| uzumaki | say I do "conn.compute.create_server(....)" and I get a huge dump of HTTP req/resp data on the terminal, I don't want that | 14:07 |
| dtantsur | uzumaki: let's move to #openstack-sdks | 14:08 |
| uzumaki | oh, there's a channel for that? | 14:08 |
| iurygregory | yeah =) | 14:09 |
| iurygregory | good morning TheJulia | 14:10 |
| uzumaki | morning iurygregory o/ long time no see (because I wasn't here) | 14:10 |
| iurygregory | morning uzumaki o/ | 14:10 |
| *** alexmcleod has quit IRC | 14:11 | |
| *** rloo has joined #openstack-ironic | 14:12 | |
| uzumaki | how you doing iurygregory ? | 14:13 |
| iurygregory | uzumaki, not 100%, my mother tested positive for covid | 14:14 |
| uzumaki | Oh, that's very unfortunate news :( hope she gets well soon | 14:15 |
| iurygregory | yeah me too, so far only the common symptoms (nothing serious) | 14:16 |
| uzumaki | Hmmm... | 14:16 |
| iurygregory | and how you doing? =) | 14:16 |
| uzumaki | I'm doing well, learning German these days | 14:17 |
| iurygregory | nice =) | 14:17 |
| * TheJulia tries to wake up | 14:18 | |
| arne_wiebalck | TheJulia: dtantsur: I had a first look at the DB queries. In each 300s power sync cycle, the DB sees 6 queries per node (2x selects from nodes, 2x from ports, 1x from portgroups, 1x from volume_connector, 1x from volume_target). In addition, there is 1x 'ROLLBACK' for each of these queries (!), and 1x 'SELECT 1' also per query. So, in total we have 18x queries per node and cycle. With 8600 nodes this is | 14:22 |
| arne_wiebalck | a rate >150000 queries in 5 mins, or 500 per sec. | 14:22 |
| TheJulia | argh | 14:23 |
| TheJulia | no wonder we're thrashing the db | 14:23 |
| iurygregory | ouch | 14:23 |
| iurygregory | poor DB | 14:23 |
| arne_wiebalck | any idea what the 'SELECT 1' is for? | 14:24 |
| TheJulia | get one, or the first record in the result set | 14:24 |
| *** alexmcleod has joined #openstack-ironic | 14:24 | |
| arne_wiebalck | my colleagues tell me this is usually to check if the client is still good | 14:24 |
| TheJulia | it is building a query set | 14:24 |
| TheJulia | I... think. | 14:25 |
| arne_wiebalck | this is only a "SELECT 1" | 14:25 |
| arne_wiebalck | no table | 14:25 |
| TheJulia | hmm | 14:25 |
| iurygregory | woot | 14:25 |
| TheJulia | Well, hmm | 14:26 |
| arne_wiebalck | http://paste.openstack.org/show/802687/ shows some lines of the trace | 14:27 |
| dtantsur | arne_wiebalck: connection check? | 14:27 |
| arne_wiebalck | dtantsur: yes, this is what my DB experts tell me | 14:27 |
| dtantsur | but yeah, we're always fetching ports, port groups and volumes for any task | 14:28 |
| dtantsur | which is a huge overkill | 14:28 |
| *** ociuhandu has quit IRC | 14:28 | |
| TheJulia | Agreed | 14:28 |
| dtantsur | so, task_manager.acquie(... node_only=True) | 14:28 |
| TheJulia | That is what I'm thinking | 14:28 |
| arne_wiebalck | if we checked the connection only say every N seconds, this would be an enormous reduction | 14:28 |
| *** alexmcleod has quit IRC | 14:28 | |
| arne_wiebalck | same for ROLLBACK | 14:28 |
| dtantsur | arne_wiebalck: I don't think we have control over that, it's a question for oslo.db/sqlalchemy | 14:29 |
| *** ociuhandu has joined #openstack-ironic | 14:29 | |
| arne_wiebalck | dtantsur: right | 14:29 |
| dtantsur | but add node_only to all periodics that don't need ports sounds reasonable | 14:29 |
| dtantsur | same likely applies to certain actions (manageable?) | 14:30 |
| TheJulia | at a minimum, if we avoid rollback somehow for periodics to at least get the list of machines, then that should help a lot on at least the main query | 14:30 |
| arne_wiebalck | rollback is also oslo.db I guess? | 14:31 |
| arne_wiebalck | this is probably to ensure the query is hitting the DB? | 14:31 |
| TheJulia | I'm trying to load an email thread that raises this question | 14:31 |
| arne_wiebalck | TheJulia: awesome, thanks | 14:31 |
| TheJulia | oh | 14:32 |
| TheJulia | you know what | 14:32 |
| TheJulia | it the translation from our use of session | 14:32 |
| arne_wiebalck | ? | 14:32 |
| TheJulia | http://lists.openstack.org/pipermail/openstack-dev/2015-September/074685.html | 14:34 |
| TheJulia | I know some of the queries are just that, queries, some of them if memory serves use a session structure which I think would cause it to roll as a full tranction with either commit/rollback | 14:34 |
| TheJulia | IF that activity is happening on the periodics, that would explain a lot | 14:35 |
| TheJulia | but I suspect the API and conductor SQL call runtimes are similar for getting the list of nodes | 14:35 |
| *** tzumainn has joined #openstack-ironic | 14:43 | |
| *** ociuhandu has quit IRC | 14:52 | |
| *** ociuhandu has joined #openstack-ironic | 14:52 | |
| Qianbiao | arne_wiebalck TheJulia i do not know the detail with Ironic, but select 1 is mostly used to keep db connection session keepalive. | 14:53 |
| *** rh-jlabarre has joined #openstack-ironic | 14:54 | |
| *** rh-jelabarre has quit IRC | 14:54 | |
| *** alexmcleod has joined #openstack-ironic | 14:54 | |
| Qianbiao | most java db management tools use it to make sure session alive | 14:55 |
| TheJulia | yeah | 14:55 |
| TheJulia | that is what I'm reading | 14:55 |
| arne_wiebalck | dtantsur: TheJulia: so the idea would be to have an additional param on the TaskManager which would skip getting ports and volume* and use this whenever we think these are not needed? | 14:56 |
| *** ociuhandu has quit IRC | 14:57 | |
| *** rh-jelabarre has joined #openstack-ironic | 15:00 | |
| *** rh-jlabarre has quit IRC | 15:00 | |
| iurygregory | I think this makes sense arne_wiebalck | 15:00 |
| Qianbiao | in web project, we program oo-db for better performance. | 15:01 |
| TheJulia | i think so | 15:01 |
| TheJulia | arne_wiebalck: ^^^ I wonder if we could actually lazy load them | 15:01 |
| *** uzumaki has quit IRC | 15:01 | |
| TheJulia | so load them when queried | 15:01 |
| Qianbiao | compose db querys into one query to reduce query counts. | 15:02 |
| arne_wiebalck | TheJulia: that sounds more sophisticated, clearly | 15:04 |
| *** ociuhandu has joined #openstack-ironic | 15:08 | |
| dtantsur | Qianbiao: oo-db? | 15:09 |
| Qianbiao | dtantsur maybe object-oriented database in english | 15:10 |
| dtantsur | interesting | 15:10 |
| dtantsur | TheJulia: lazy loading attributes.. sounds like we're not using sqlalchemy right | 15:11 |
| dtantsur | because one of its features is lazy loading | 15:11 |
| dtantsur | (this is old man yelling at cloud, not a call for actions) | 15:11 |
| Qianbiao | there are too many user in website, so database first programming is required at begining of a new project. | 15:12 |
| dtantsur | otherwise, yes, I think lazy loading for ports and co is a good idea IMO | 15:12 |
| *** moshiur has quit IRC | 15:12 | |
| dtantsur | Qianbiao: interesting. it reminds me of my old times at Skype: we implemented half of the logic in postgresql stored procedures :) | 15:13 |
| Qianbiao | :) | 15:13 |
| *** moshiur has joined #openstack-ironic | 15:13 | |
| * TheJulia hands dtantsur some grass so he can yell at the cloud in proper style | 15:14 | |
| Qianbiao | in china, sqlalchemy, hibernate is not suggested to use. | 15:15 |
| Qianbiao | programmer must control all sql themselves. | 15:15 |
| Qianbiao | like if you only want column1 and column2, then column3 should not present in select | 15:15 |
| *** moshiur has quit IRC | 15:15 | |
| dtantsur | I can relate. sqlalchemy is too much magic to my taste | 15:16 |
| dtantsur | I think there are lighter ORM though | 15:16 |
| *** moshiur has joined #openstack-ironic | 15:16 | |
| Qianbiao | yes, tools can mapping result to object, but should not generate sql. | 15:16 |
| dtantsur | Well.. there is a downside of making everyone write an SQL: you have to be pretty good in it | 15:17 |
| dtantsur | (not that ORM's are always good at it..) | 15:17 |
| Qianbiao | yes | 15:17 |
| dtantsur | I kind of like this approach: https://docs.ponyorm.org/firststeps.html#writing-queries where you write queries on a pythonic way | 15:19 |
| dtantsur | (I'm not familiar with ponyORM, just one of the first links in the search) | 15:20 |
| dtantsur | I think what we do in ironic is the worst from the two worlds: | 15:21 |
| dtantsur | we use a heavy-weight ORM, but we do joins manually in Python | 15:21 |
| iurygregory | ouch =X | 15:22 |
| dtantsur | yep | 15:23 |
| ajya | if want to get better performance then have to write your own SQL, otherwise it is what it is. Some ORMs handle SQL generation better, but in the end there is always some case where it's not good enough :) | 15:23 |
| iurygregory | joins are always expensive | 15:23 |
| dtantsur | to be big extent because of our usage of oslo Object's | 15:23 |
| dtantsur | iurygregory: databases are pretty good at them; python - not so much | 15:23 |
| iurygregory | yeah | 15:23 |
| dtantsur | as discovered by arne_wiebalck, TaskManager is a redundant python-based joiner :) | 15:24 |
| rpittau | lol | 15:24 |
| *** rcernin has joined #openstack-ironic | 15:34 | |
| *** rcernin has quit IRC | 15:39 | |
| *** moshiur has quit IRC | 15:39 | |
| TheJulia | We somehow need to benchmark our db interactions | 15:39 |
| openstackgerrit | Dmitry Tantsur proposed openstack/python-ironicclient master: Support YAML files wherever JSON files are accepted https://review.opendev.org/c/openstack/python-ironicclient/+/775861 | 15:39 |
| dtantsur | TheJulia++ | 15:39 |
| TheJulia | arne_wiebalck: by chance are you emitting metrics to the message bus? | 15:49 |
| *** uzumaki has joined #openstack-ironic | 15:51 | |
| arne_wiebalck | TheJulia: we collect a lot of metrics and all logs ... what metric are you thinking of? | 15:51 |
| TheJulia | ConductorManager._sync_power_states | 15:54 |
| arne_wiebalck | IIUC, the idea is we should pursue both, reduce the overall number of queries and optimise them, is that right? | 15:55 |
| TheJulia | I think so yes | 15:55 |
| TheJulia | and ConductorManager._sync_local_state | 15:55 |
| * dtantsur just had an awkward moment until he realized that "on" in YAML means True | 15:56 | |
| TheJulia | heh | 15:56 |
| arne_wiebalck | TheJulia: I need to check, we probably do, but I do not use them atm. | 15:56 |
| TheJulia | the queries are a little different, the delta between those two *should* give us an idea | 15:57 |
| openstackgerrit | Dmitry Tantsur proposed openstack/ironic-python-agent master: New deploy step for injecting arbitrary files https://review.opendev.org/c/openstack/ironic-python-agent/+/775428 | 15:57 |
| dtantsur | as a result I won't use "on" as a key in ^^^ :) | 15:57 |
| TheJulia | at least, an idea as a starting point | 15:57 |
| TheJulia | That will tell us where we may want to focus our braincells first | 15:58 |
| *** rpittau is now known as rpittau|afk | 16:10 | |
| arne_wiebalck | TheJulia: no, we do not | 16:13 |
| arne_wiebalck | TheJulia: I could look into enabling this, though | 16:14 |
| TheJulia | dtantsur: do you know if the ansible maintainers know that the the linting job is broken due to setuptools_rust ? | 16:14 |
| arne_wiebalck | TheJulia: should be more or less straight-forward to integrate this into our monitoring infra | 16:14 |
| dtantsur | TheJulia: they made it non-voting | 16:14 |
| TheJulia | dtantsur: ack | 16:14 |
| *** gyee has joined #openstack-ironic | 16:27 | |
| *** Qianbiao has quit IRC | 16:28 | |
| iurygregory | for anyone interested we are going to debug the privsep issues now on https://bluejeans.com/5108563600/ | 16:30 |
| *** hjensas is now known as hjensas|afk | 16:36 | |
| *** alexmcleod has quit IRC | 16:36 | |
| arne_wiebalck | dtantsur: TheJulia: to reduce the number of queries, what should be the path to follow? | 16:37 |
| arne_wiebalck | - have an option on TaskManager to not collect all data on all queries? (acc. to my math, if only nodes are needed, this should reduce the number of DB queries by two thirds) | 16:37 |
| arne_wiebalck | - look into sqlalchemy lazy loading? | 16:37 |
| arne_wiebalck | - see if the reset_on_return sqlalchemy parameter could be changed? | 16:37 |
| dtantsur | arne_wiebalck: let's start with lazy loading ports/portgroups/volumes? | 16:37 |
| dtantsur | manually, in Python, I mean? | 16:38 |
| dtantsur | changing sqlalchemy will also require changing the object layer | 16:38 |
| arne_wiebalck | dtantsur: do we do this already somewhere? lazy loading I mean | 16:39 |
| dtantsur | I don't think so, but should not be hard | 16:39 |
| arne_wiebalck | dtantsur: it should have a similar effect as the parameter | 16:39 |
| *** ociuhandu has quit IRC | 16:41 | |
| *** ociuhandu has joined #openstack-ironic | 16:41 | |
| TheJulia | PYOPTIMIZE_TINYIPA | 16:42 |
| TheJulia | iurygregory: ^^^ | 16:42 |
| iurygregory | ack | 16:43 |
| *** ociuhandu has quit IRC | 16:44 | |
| *** ociuhandu has joined #openstack-ironic | 16:44 | |
| openstackgerrit | Verification of a change to openstack/ironic-inspector failed: Fix memcached host address. https://review.opendev.org/c/openstack/ironic-inspector/+/775640 | 16:46 |
| TheJulia | so we figured out why the privsep stuff is failing randomly | 16:51 |
| TheJulia | it is the jobs on rax | 16:51 |
| openstackgerrit | Iury Gregory Melo Ferreira proposed openstack/ironic-lib master: Add initial privsep structure https://review.opendev.org/c/openstack/ironic-lib/+/745536 | 16:51 |
| dtantsur | TheJulia: rax again? wow | 16:55 |
| TheJulia | yup | 16:55 |
| TheJulia | it is tinycore + pycparser which gets pulled in because the have optimized builds with tinycore | 16:56 |
| TheJulia | and we override and use tinycore on rax because of image size and rax cpu performance | 16:56 |
| *** alexmcleod has joined #openstack-ironic | 16:58 | |
| iurygregory | rax love us <3 | 17:00 |
| iurygregory | and we probably hate rax =D | 17:01 |
| TheJulia | heh | 17:01 |
| TheJulia | I'm testing image sizes regarding not doing the optomized image generation | 17:01 |
| TheJulia | so... yay for time it will take :( | 17:01 |
| JayF | I've had to resist several times to take potshots at rax in here this morning :P | 17:05 |
| *** rloo has quit IRC | 17:09 | |
| *** lucasagomes has quit IRC | 17:09 | |
| *** rloo has joined #openstack-ironic | 17:09 | |
| *** rloo has quit IRC | 17:12 | |
| *** rloo has joined #openstack-ironic | 17:12 | |
| TheJulia | There we go, tinyipa.gz is 61M, kernel is 4.9 | 17:19 |
| * TheJulia re-does it with the optimization code stripped out | 17:21 | |
| *** ociuhandu has quit IRC | 17:30 | |
| *** ociuhandu has joined #openstack-ironic | 17:31 | |
| *** ociuhandu has quit IRC | 17:31 | |
| *** dougsz has quit IRC | 17:31 | |
| *** ociuhandu has joined #openstack-ironic | 17:31 | |
| iurygregory | woot I can't find 745536 in zuul.openstack.org O.o | 17:33 |
| *** rcernin has joined #openstack-ironic | 17:34 | |
| TheJulia | did results already get posted? | 17:38 |
| TheJulia | 66mb... i think we can loose it | 17:38 |
| *** rcernin has quit IRC | 17:39 | |
| dtantsur | folks, if you review https://review.opendev.org/c/openstack/ironic-python-agent/+/775428 I promise to write a detailed blog post on how to use it :) | 17:43 |
| TheJulia | heh | 17:44 |
| arne_wiebalck | bye everyone o/ | 17:47 |
| iurygregory | hum 5mb is ok | 17:49 |
| openstackgerrit | Julia Kreger proposed openstack/ironic-python-agent-builder master: Remove PYOPTIMIZE_TINYIPA capability https://review.opendev.org/c/openstack/ironic-python-agent-builder/+/775884 | 17:54 |
| openstackgerrit | Julia Kreger proposed openstack/ironic-python-agent-builder master: Remove PYOPTIMIZE_TINYIPA capability https://review.opendev.org/c/openstack/ironic-python-agent-builder/+/775884 | 17:54 |
| iurygregory | TheJulia, should I try to use Depends-On? | 17:56 |
| TheJulia | you can | 17:56 |
| TheJulia | try it that is | 17:56 |
| TheJulia | if you want. It seems to be fine, but we'll see what CI says | 17:57 |
| dtantsur | depends-on IPA-builder only works for jobs that build images (ironic-lib jobs do) | 17:57 |
| JayF | hmm, I knew we had talked about putting deploy steps in IPA-proper, I didn't realize we were going to put them in the GenericHardwareManager | 17:57 |
| *** ociuhandu_ has joined #openstack-ironic | 17:57 | |
| *** uzumaki has quit IRC | 17:57 | |
| JayF | I guess that's OK? I don't know why it wouldn't be, just was surprised when I looked at that patch | 17:57 |
| dtantsur | JayF: any built-in hardware manager will be equivalent, right? because they all are loaded, and admin has no say in it. | 17:58 |
| *** uzumaki has joined #openstack-ironic | 17:58 | |
| JayF | I was thinking something more like evaluate_hardware_support() looking for the config/args of files to inject, in this case | 17:58 |
| JayF | but I think the way you did it is probably better... | 17:58 |
| dtantsur | evaluate_hardware_support has no access to the node or the deploy steps | 17:59 |
| JayF | yeah, which is why your way is right, because that all runs before even ironic is talking to ipa | 17:59 |
| JayF | I guess I wish we had more categorization in there rather than a single large "junk drawer"-style class | 17:59 |
| TheJulia | Review jam time? | 17:59 |
| JayF | I'm going to join | 18:00 |
| *** ociuhandu has quit IRC | 18:01 | |
| *** ociuhandu_ has quit IRC | 18:02 | |
| *** rloo has quit IRC | 18:09 | |
| *** rloo has joined #openstack-ironic | 18:14 | |
| openstackgerrit | Iury Gregory Melo Ferreira proposed openstack/ironic-lib master: Add initial privsep structure https://review.opendev.org/c/openstack/ironic-lib/+/745536 | 18:15 |
| *** k_mouza has quit IRC | 18:28 | |
| *** derekh has quit IRC | 18:32 | |
| openstackgerrit | Dmitry Tantsur proposed openstack/python-ironicclient master: Clearer error message when unable to parse JSON https://review.opendev.org/c/openstack/python-ironicclient/+/775889 | 18:39 |
| openstackgerrit | Arun S A G proposed openstack/ironic master: Add anaconda support in the pxe boot driver https://review.opendev.org/c/openstack/ironic/+/775891 | 18:59 |
| *** uzumaki has quit IRC | 19:12 | |
| *** dtantsur is now known as dtantsur|afk | 19:20 | |
| dtantsur|afk | o/ | 19:20 |
| iurygregory | I love agent token ronic_python_agent.errors.HeartbeatError: Error heartbeating to agent API: Error 400: Invalid or missing agent token received. | 19:25 |
| iurygregory | D: | 19:25 |
| iurygregory | and ofc it failed in the same job \o/ | 19:26 |
| *** uzumaki has joined #openstack-ironic | 19:27 | |
| stevebaker | morning | 19:29 |
| TheJulia | good morning | 19:29 |
| openstackgerrit | Julia Kreger proposed openstack/ironic-python-agent-builder master: Remove PYOPTIMIZE_TINYIPA capability https://review.opendev.org/c/openstack/ironic-python-agent-builder/+/775884 | 19:31 |
| iurygregory | good morning stevebaker | 19:34 |
| stevebaker | TheJulia: could I talk to you about the indicator API? I think there is 2 things going on. 1) the implementation diverted from the spec, different endpoints and behaviour. Also 2) the implementation is incomplete so it doesn't quite work anyway. | 19:35 |
| stevebaker | iurygregory: hey | 19:35 |
| *** rcernin has joined #openstack-ironic | 19:35 | |
| iurygregory | o/ | 19:35 |
| TheJulia | stevebaker: ugh | 19:37 |
| TheJulia | stevebaker: do you want to just talk about it early on our bug triage call? | 19:37 |
| stevebaker | TheJulia: good idea | 19:38 |
| iurygregory | I'm the one who added the documentation https://docs.openstack.org/api-ref/baremetal/#indicators-management based on the spec =X | 19:38 |
| iurygregory | so if something needs update there let me know | 19:38 |
| stevebaker | iurygregory: will do, thanks | 19:39 |
| *** rcernin has quit IRC | 19:40 | |
| *** ayoung has joined #openstack-ironic | 19:45 | |
| *** tosin has quit IRC | 19:48 | |
| openstackgerrit | Dmitry Tantsur proposed openstack/ironic master: Validate configdrive string format https://review.opendev.org/c/openstack/ironic/+/775922 | 19:48 |
| openstackgerrit | Dmitry Tantsur proposed openstack/ironic master: Use configdrive_use_object_store when swift is enabled https://review.opendev.org/c/openstack/ironic/+/775924 | 19:51 |
| TheJulia | dtantsur|afk: double checking your recollection, a user shouldn't be able to patch subresources like ports/portgroups.. right? or wrong? | 20:08 |
| TheJulia | hmmm | 20:10 |
| TheJulia | looks like it is for get only | 20:15 |
| TheJulia | which matches my memory | 20:15 |
| TheJulia | stevebaker: I've joined the call early if you want to hop on | 20:20 |
| *** k_mouza has joined #openstack-ironic | 20:29 | |
| *** k_mouza has quit IRC | 20:34 | |
| *** hoonetorg has quit IRC | 21:02 | |
| *** rcernin has joined #openstack-ironic | 21:20 | |
| *** rcernin has quit IRC | 21:29 | |
| *** rcernin has joined #openstack-ironic | 21:34 | |
| *** rcernin has quit IRC | 21:39 | |
| *** hoonetorg has joined #openstack-ironic | 21:52 | |
| *** rcernin has joined #openstack-ironic | 22:24 | |
| *** dtruong has quit IRC | 22:37 | |
| *** dtruong has joined #openstack-ironic | 22:37 | |
| iurygregory | yay the error is different \o/ | 22:38 |
| iurygregory | https://zuul.opendev.org/t/openstack/build/a0e8b02112144d2d8dd64c2bf5ba4a10/log/controller/logs/screen-ir-cond.txt?severity=4 | 22:38 |
| iurygregory | TheJulia, now at least I have an error that makes sense | 22:40 |
| * iurygregory goes for dinner brb | 22:40 | |
| TheJulia | woot | 22:40 |
| TheJulia | stevebaker: hey, shouldn't get_all get called if I ask for a url GET: /v1/nodes/38d5abed-c585-4fce-a57e-a2ffc2a2ec6f/portgroups ? well, get_all on the portgroup subcontroller | 22:41 |
| janders | good morning Ironic o/ | 22:43 |
| mgagne | I'm trying to enable TLS for json-rpc. I configured json_rpc/use_ssl, json_rpc/certfile and json_rpc/keyfile but I'm getting this error when starting conductor. Any idea? ValueError: certfile must be specified for server-side operations | 22:44 |
| mgagne | Answering my own question, cert needs to be in ssl/cert_file and ssl/key_file. The other configs are for the client certificate. | 22:47 |
| stevebaker | TheJulia: (looking) | 23:01 |
| stevebaker | TheJulia: I would assume so, yes. But I suspect the implementation is incorrectly using the get_all node argument instead of self.parent_node_ident | 23:08 |
| TheJulia | Sigh, that was my feeling. I guess more refactoring is needed than I thought for the rbac stuff in portgroups :( | 23:12 |
| TheJulia | Something for a fresh mind tomorrow | 23:13 |
| *** rloo has quit IRC | 23:46 | |
| stevebaker | TheJulia: oh wait, portgroup.py#119 is where it happens | 23:54 |
| iurygregory | morning janders o/ | 23:57 |
| *** tosky has quit IRC | 23:57 | |
Generated by irclog2html.py 2.17.2 by Marius Gedminas - find it at https://mg.pov.lt/irclog2html/!