| *** penick has quit IRC | 00:00 | |
| *** smoriya_ has joined #openstack-ironic | 00:01 | |
| *** rajathagasthya has quit IRC | 00:03 | |
| *** sacharya has quit IRC | 00:04 | |
| *** penick has joined #openstack-ironic | 00:07 | |
| *** rajathagasthya has joined #openstack-ironic | 00:08 | |
| *** naohirot has joined #openstack-ironic | 00:09 | |
| *** chlong has joined #openstack-ironic | 00:17 | |
| *** penick has quit IRC | 00:27 | |
| *** devananda has quit IRC | 00:28 | |
| *** amotoki has joined #openstack-ironic | 00:28 | |
| *** Pavo has joined #openstack-ironic | 00:36 | |
| *** Pavo has left #openstack-ironic | 00:39 | |
| *** rajathagasthya has quit IRC | 00:40 | |
| *** rajathagasthya has joined #openstack-ironic | 00:42 | |
| *** amotoki has quit IRC | 00:44 | |
| *** jaybeale has quit IRC | 00:45 | |
| *** devananda has joined #openstack-ironic | 00:53 | |
| *** Sukhdev has joined #openstack-ironic | 00:55 | |
| *** Sukhdev has quit IRC | 00:59 | |
| *** sacharya has joined #openstack-ironic | 01:02 | |
| *** sacharya has quit IRC | 01:09 | |
| *** penick has joined #openstack-ironic | 01:13 | |
| lintan | morning mrda and NobodyCam | 01:20 |
|---|---|---|
| *** _zouyee has quit IRC | 01:23 | |
| *** penick has quit IRC | 01:31 | |
| *** kkoski has joined #openstack-ironic | 01:31 | |
| *** mtanino has quit IRC | 01:32 | |
| *** baoli_ has quit IRC | 01:34 | |
| *** zhenguo has joined #openstack-ironic | 01:37 | |
| openstackgerrit | Tan Lin proposed openstack/ironic: Follow-up Patch on I4d81001c01e7908100a6571b366cb296253f2fc1 https://review.openstack.org/252883 | 01:39 |
| mrda | hey lintan | 01:51 |
| *** sacharya has joined #openstack-ironic | 01:54 | |
| *** Haomeng has quit IRC | 02:00 | |
| *** _zouyee has joined #openstack-ironic | 02:02 | |
| *** sacharya has quit IRC | 02:05 | |
| *** rajathagasthya has quit IRC | 02:10 | |
| *** sinval has quit IRC | 02:11 | |
| *** amotoki has joined #openstack-ironic | 02:13 | |
| *** aginwala has quit IRC | 02:21 | |
| *** changzhi has joined #openstack-ironic | 02:29 | |
| openstackgerrit | Zhenguo Niu proposed openstack/ironic: Add console feature to ssh driver https://review.openstack.org/241453 | 02:31 |
| *** aginwala has joined #openstack-ironic | 02:35 | |
| *** hanchao has joined #openstack-ironic | 02:35 | |
| *** aginwala has quit IRC | 02:36 | |
| *** yonglihe has quit IRC | 02:41 | |
| *** baoli has joined #openstack-ironic | 02:44 | |
| changzhi | hi, all. Did anyone deploy bare metal machines by using Ironic? | 02:47 |
| openstackgerrit | Merged openstack/ironic: Correct the db connection string in dev-quickstart https://review.openstack.org/250594 | 02:49 |
| openstackgerrit | Naohiro Tamura proposed openstack/ironic-specs: Enhance Power Interface for Soft Power Off and Inject NMI https://review.openstack.org/186700 | 02:52 |
| hanchao | hello, did anyone deployed the ironic service on a individual node rather than located same on the controller node? | 02:56 |
| *** gbraad has joined #openstack-ironic | 02:57 | |
| *** rloo has quit IRC | 03:00 | |
| openstackgerrit | Yuiko Takada proposed openstack/ironic-specs: Add allowed transitions to API https://review.openstack.org/224022 | 03:19 |
| *** thrash has quit IRC | 03:24 | |
| *** Piet has joined #openstack-ironic | 03:31 | |
| *** Nisha has joined #openstack-ironic | 03:34 | |
| *** e0ne_ has joined #openstack-ironic | 03:52 | |
| *** links has joined #openstack-ironic | 04:00 | |
| *** e0ne_ has quit IRC | 04:00 | |
| *** Nisha has quit IRC | 04:04 | |
| *** baoli has quit IRC | 04:08 | |
| *** e0ne has joined #openstack-ironic | 04:12 | |
| *** e0ne has quit IRC | 04:16 | |
| *** links has quit IRC | 04:23 | |
| *** Nisha has joined #openstack-ironic | 04:31 | |
| *** jaybeale has joined #openstack-ironic | 04:45 | |
| *** jaybeale has quit IRC | 04:59 | |
| *** amotoki_ has joined #openstack-ironic | 05:09 | |
| *** links has joined #openstack-ironic | 05:09 | |
| openstackgerrit | ChangBo Guo(gcb) proposed openstack/ironic: test:clean up wrong value in method CONF.set_override https://review.openstack.org/243430 | 05:10 |
| *** jaybeale has joined #openstack-ironic | 05:10 | |
| *** amotoki_ has quit IRC | 05:12 | |
| *** amotoki has quit IRC | 05:13 | |
| *** Haomeng has joined #openstack-ironic | 05:16 | |
| *** amotoki has joined #openstack-ironic | 05:17 | |
| *** amotoki has quit IRC | 05:17 | |
| *** mayflyman has joined #openstack-ironic | 05:18 | |
| *** jaybeale has quit IRC | 05:22 | |
| *** aginwala has joined #openstack-ironic | 05:34 | |
| *** Sukhdev has joined #openstack-ironic | 05:39 | |
| *** Sukhdev has quit IRC | 05:43 | |
| *** hoangcx has joined #openstack-ironic | 05:53 | |
| openstackgerrit | Merged openstack/bifrost: Adjusted line length to 79 characters https://review.openstack.org/251544 | 06:03 |
| *** rcernin has joined #openstack-ironic | 06:13 | |
| *** mayflyman has quit IRC | 06:14 | |
| *** Sukhdev has joined #openstack-ironic | 06:19 | |
| *** jaybeale has joined #openstack-ironic | 06:22 | |
| *** Haomeng has quit IRC | 06:22 | |
| *** thanhnt-z has joined #openstack-ironic | 06:28 | |
| *** ChubYann has quit IRC | 06:30 | |
| *** amotoki has joined #openstack-ironic | 06:30 | |
| *** aginwala has quit IRC | 06:35 | |
| *** aginwala has joined #openstack-ironic | 06:37 | |
| *** jlk has joined #openstack-ironic | 06:41 | |
| *** jlk has quit IRC | 06:41 | |
| *** jlk has joined #openstack-ironic | 06:41 | |
| *** Haomeng|2 has joined #openstack-ironic | 06:49 | |
| *** jaybeale has quit IRC | 06:50 | |
| *** Haomeng has joined #openstack-ironic | 06:50 | |
| *** rcernin has quit IRC | 06:59 | |
| *** PradeepV has joined #openstack-ironic | 07:04 | |
| *** deray has joined #openstack-ironic | 07:04 | |
| *** rcernin has joined #openstack-ironic | 07:07 | |
| *** absubram has quit IRC | 07:15 | |
| *** chlong has quit IRC | 07:19 | |
| *** Sukhdev has quit IRC | 07:23 | |
| *** kozhukalov_ has joined #openstack-ironic | 07:29 | |
| openstackgerrit | Tan Lin proposed openstack/ironic: Add a config option [upgrade_level]/condcutor_rpcapi_cap https://review.openstack.org/253355 | 07:31 |
| openstackgerrit | Naohiro Tamura proposed openstack/ironic: Update python-scciclient version number https://review.openstack.org/253358 | 07:34 |
| *** kozhukalov_ has quit IRC | 07:34 | |
| sergek | Morning, Ironicers 😃 | 07:46 |
| *** _zouyee has quit IRC | 07:47 | |
| lintan | morning sergek :) | 07:47 |
| sergek | lintan: o/ | 07:47 |
| thanhnt-z | Morning :D | 07:48 |
| *** kozhukalov_ has joined #openstack-ironic | 07:48 | |
| openstackgerrit | Tan Lin proposed openstack/ironic: Correct NotImplemented to NotImplementedError in rpcapi.py https://review.openstack.org/253365 | 07:50 |
| thanhnt-z | Hi Haomeng | 07:51 |
| openstackgerrit | Kan proposed openstack/python-ironicclient: [WIP]Format the valid fields for list commands https://review.openstack.org/245541 | 07:58 |
| *** kozhukalov_ has quit IRC | 08:03 | |
| openstackgerrit | Tan Lin proposed openstack/ironic: Add a config option [upgrade_level]/condcutor_rpcapi_cap https://review.openstack.org/253355 | 08:06 |
| *** changzhi has quit IRC | 08:06 | |
| openstackgerrit | Nisha Agarwal proposed openstack/python-ironicclient: Add CLI support for RAID configuration https://review.openstack.org/226234 | 08:10 |
| Haomeng|2 | thanhnt-z: morning:) | 08:11 |
| Haomeng|2 | sergek: morning:) | 08:12 |
| thanhnt-z | Haomeng: related to the issue that I got some days ago, it was already reported as this bug: https://bugs.launchpad.net/ironic/+bug/1494605 | 08:13 |
| openstack | Launchpad bug 1494605 in Ironic "shellinabox doesn't restart when ironic-conductor restarted" [Low,Triaged] - Assigned to takahiko nishiyama (tnishiya) | 08:13 |
| Haomeng|2 | thanhnt-z: ok, let me check, thanks for your bug | 08:13 |
| *** Haomeng|2 is now known as Haomeng | 08:14 | |
| thanhnt-z | Haomeng: not mine :D, it was reported some months ago :| | 08:14 |
| sergek | Haomeng: o/ | 08:14 |
| Haomeng | thanhnt-z: ok, dont worry, but it is assigned to submitter already | 08:14 |
| Haomeng | thanhnt-z: so understand your concern that you have same issue, and is how about the progress? | 08:15 |
| thanhnt-z | Haomeng: Yep, when I set node-set-console-mode is True, the shellinabox service is not restarted and not run this command: shellinaboxd -t -p 10101 --background=/tmp/9e3c0320-04f1-4175-8a2d-4be2c91556fd.pid -s /:1000:1000:HOME:ipmitool -H baremetal_ip -I lanplus -U admin -f /tmp/9e3c0320-04f1-4175-8a2d-4be2c91556fd.pw -v sol activate | 08:18 |
| thanhnt-z | Haomeng: so I have to run this command by manually | 08:18 |
| *** kozhukalov_ has joined #openstack-ironic | 08:19 | |
| *** athomas has joined #openstack-ironic | 08:19 | |
| *** jaybeale has joined #openstack-ironic | 08:20 | |
| *** deray has quit IRC | 08:20 | |
| Haomeng | thanhnt-z: ok, so can you enable debug log and send me(via gmail?) the conductor's log, and let me check what happens when enabling the console? | 08:20 |
| thanhnt-z | Haomeng: But it did not orcuss on my co-worker's environment. So that's why I told that it seems not a Ironic Bug | 08:20 |
| thanhnt-z | Haomeng: sure that conductor-log got no error message | 08:21 |
| *** romcheg2 has quit IRC | 08:21 | |
| Haomeng | thanhnt-z: if we run node-set-console-mode is true, and it returns as normal behavior, but no shellinabox process, it should be a bug | 08:21 |
| Haomeng | thanhnt-z: ok, but I have no env to recreate now, so can you help to provide more details, such as the shellinabox process status | 08:23 |
| *** romcheg1 has joined #openstack-ironic | 08:23 | |
| Haomeng | thanhnt-z: ps -ef|grep shellinabox | 08:23 |
| Haomeng | thanhnt-z: if there is no such process at all, have to check conductor log and debug the code | 08:23 |
| *** vsaienko has joined #openstack-ironic | 08:24 | |
| vsaienko | Hello Ironic team, could you please have look at the devstack patches that enables ironic neutron integration testing | 08:25 |
| *** MattMan has quit IRC | 08:25 | |
| Haomeng | thanhnt-z: got this log in code - https://github.com/openstack/ironic/blob/ee0987000c694a4317d1979a81accb659c196eda/ironic/drivers/modules/console_utils.py#L211 | 08:25 |
| *** MattMan has joined #openstack-ironic | 08:25 | |
| Haomeng | thanhnt-z: so can you make sure if your log is in debug level ,and help to grep 'Running subprocess:' | 08:25 |
| thanhnt-z | Haomeng: Yep, I will send you an email related to this concern in next Monday, it's ok? Now I also dont have enviroment :| and I will rebuild on next monday | 08:26 |
| vsaienko | https://review.openstack.org/#/c/247513/ | 08:26 |
| vsaienko | https://review.openstack.org/#/c/248048/ | 08:26 |
| vsaienko | https://review.openstack.org/#/c/249717/ | 08:26 |
| vsaienko | https://review.openstack.org/#/c/248074/ | 08:26 |
| vsaienko | https://review.openstack.org/#/c/250906/ | 08:26 |
| Haomeng | thanhnt-z: ok, will check your mail, thank you | 08:26 |
| thanhnt-z | Haomeng: Sure that it set DEBUG level | 08:26 |
| Haomeng | thanhnt-z: assume you enable log in debug level | 08:26 |
| Haomeng | thanhnt-z: ok, that is fine | 08:26 |
| Haomeng | thanhnt-z: inform me once you sent mail | 08:26 |
| thanhnt-z | Haomeng: sure | 08:27 |
| Haomeng | thanhnt-z: will be next Mon, right? ok | 08:27 |
| thanhnt-z | Haomeng: yep | 08:27 |
| Haomeng | thanhnt-z: ok | 08:27 |
| Haomeng | thanhnt-z: but if there is no env to be recreated, it is diffcult to debug I think, will try:) | 08:28 |
| thanhnt-z | Haomeng: yes, I see. I tried to debug in that way some times before :D | 08:29 |
| Haomeng | thanhnt-z: :) | 08:30 |
| thanhnt-z | Haomeng: have you tried to investigate about how to use nova console for Ironic node yet? | 08:32 |
| *** ifarkas has joined #openstack-ironic | 08:32 | |
| Haomeng | thanhnt-z: no, and think it is not easy to support both nova novnc and ironic terminal console | 08:32 |
| Haomeng | thanhnt-z: but maybe it can be integrated in horizon, need to horizon team guys support, ironic just pass the console url | 08:33 |
| *** deray has joined #openstack-ironic | 08:33 | |
| thanhnt-z | Haomeng: yes, it is not easy issue, I have just wanted to investigate more about them. Then, I will have a better look about ironic | 08:35 |
| Haomeng | thanhnt-z: yes | 08:35 |
| *** mgoddard has joined #openstack-ironic | 08:47 | |
| *** jcoufal has joined #openstack-ironic | 08:49 | |
| *** _degorenko|afk is now known as degorenko | 09:03 | |
| openstackgerrit | Tan Lin proposed openstack/ironic: Add Code Contribution Guide for Ironic https://review.openstack.org/248647 | 09:03 |
| *** dims_ has quit IRC | 09:05 | |
| *** jaybeale has quit IRC | 09:05 | |
| *** praneshp has quit IRC | 09:10 | |
| *** dims has joined #openstack-ironic | 09:11 | |
| *** mbound_ has joined #openstack-ironic | 09:13 | |
| openstackgerrit | Tan Lin proposed openstack/ironic: Follow-up Patch on 1ab3c9217edc8ebed7643f8ded7b16c39dd4bc70 https://review.openstack.org/252883 | 09:14 |
| openstackgerrit | Yuriy Zveryanskyy proposed openstack/ironic: Fix awake AMT unit test https://review.openstack.org/253402 | 09:21 |
| *** yuanying has quit IRC | 09:22 | |
| yuriyz | hi Ironic | 09:22 |
| *** aginwala has quit IRC | 09:23 | |
| *** dims has quit IRC | 09:23 | |
| lintan | hi yuriyz :) | 09:25 |
| yuriyz | hi lintan haomeng, more AMT stuff :) https://review.openstack.org/#/c/253402/ | 09:26 |
| lintan | cool, yuriyz, I will try your patch :) | 09:29 |
| -openstackstatus- NOTICE: Tox tests are broken at the moment. From openstack-infra we are working to fix them. Please don't approve changes until we notify that tox tests work again. | 09:31 | |
| *** dims has joined #openstack-ironic | 09:34 | |
| *** dtantsur|afk is now known as dtantsur | 09:35 | |
| dtantsur | Morning Ironic! | 09:35 |
| dtantsur | gate is broken, meaning... | 09:35 |
| dtantsur | ...TGIF! | 09:35 |
| *** gbraad has quit IRC | 09:35 | |
| *** derekh has joined #openstack-ironic | 09:38 | |
| openstackgerrit | Tan Lin proposed openstack/python-ironicclient: Correct node-port-list help info https://review.openstack.org/246729 | 09:39 |
| *** ionutbalutoiu has joined #openstack-ironic | 09:45 | |
| yuriyz | morning dtantsur | 09:46 |
| dtantsur | o/ | 09:47 |
| *** mdbooth has quit IRC | 09:47 | |
| aarefiev | morning all! | 09:50 |
| yuriyz | dtantsur, looks like I'm member of ironic-inspector-specs cores | 09:51 |
| dtantsur | yuriyz, well, as all ironic-core's | 09:51 |
| dtantsur | ironic-inspector-core contains ironic-core | 09:51 |
| *** mdbooth has joined #openstack-ironic | 09:51 | |
| dtantsur | usual rule applies: don't approve if you don't feel you should :) | 09:52 |
| *** Marga_ has quit IRC | 09:54 | |
| yuriyz | datntsur good maybe I will look into Inspector, I prefer "-1" for reviews :) | 09:54 |
| *** mkovacik_ has joined #openstack-ironic | 09:55 | |
| sergek | o/ | 09:55 |
| *** zhenguo has quit IRC | 09:59 | |
| *** kevinbenton has quit IRC | 10:01 | |
| *** naohirot has quit IRC | 10:02 | |
| openstackgerrit | Tan Lin proposed openstack/ironic: Follow-up Patch on 1ab3c9217edc8ebed7643f8ded7b16c39dd4bc70 https://review.openstack.org/252883 | 10:02 |
| *** kevinbenton has joined #openstack-ironic | 10:04 | |
| dtantsur | yuriyz, :) | 10:07 |
| dtantsur | morning sergek | 10:07 |
| *** Marga_ has joined #openstack-ironic | 10:07 | |
| *** Marga_ has quit IRC | 10:11 | |
| *** Marga_ has joined #openstack-ironic | 10:11 | |
| *** athomas has quit IRC | 10:15 | |
| *** pas-ha has joined #openstack-ironic | 10:19 | |
| *** hoangcx has quit IRC | 10:21 | |
| *** athomas has joined #openstack-ironic | 10:24 | |
| *** Marga_ has quit IRC | 10:28 | |
| *** yuanying has joined #openstack-ironic | 10:29 | |
| *** openstackgerrit has quit IRC | 10:32 | |
| *** yuanying has quit IRC | 10:32 | |
| *** yuanying has joined #openstack-ironic | 10:33 | |
| *** openstackgerrit has joined #openstack-ironic | 10:33 | |
| *** yuanying has quit IRC | 10:33 | |
| *** yuanying has joined #openstack-ironic | 10:34 | |
| *** electrofelix has joined #openstack-ironic | 10:35 | |
| openstackgerrit | Merged openstack/python-ironic-inspector-client: Fixed links to the new ironic-inspector documentation https://review.openstack.org/252984 | 10:44 |
| sambetts-afk | Morning all o/ | 10:44 |
| dtantsur | morning sambetts-afk | 10:46 |
| *** sambetts-afk is now known as sambetts | 10:46 | |
| sambetts | o/ dtantsur hows it going? | 10:50 |
| dtantsur | it's Friday :) | 10:50 |
| sambetts | haha yes :D TGIF | 10:50 |
| openstackgerrit | Shuquan Huang proposed openstack/ironic: Move install-guide's troubleshooting to troubleshooting page https://review.openstack.org/252371 | 10:50 |
| *** aginwala has joined #openstack-ironic | 10:54 | |
| *** PradeepV has quit IRC | 10:55 | |
| *** thanhnt-z has quit IRC | 10:57 | |
| *** aginwala has quit IRC | 10:58 | |
| *** deray has quit IRC | 11:09 | |
| openstackgerrit | Vladyslav Drok proposed openstack/ironic: Fix awake AMT unit test https://review.openstack.org/253402 | 11:11 |
| lucasagomes | morning all! | 11:13 |
| sambetts | o/ lucasagomes! | 11:13 |
| dtantsur | morning lucasagomes | 11:14 |
| dtantsur | folks, if you haven't seen the notice, tox jobs are broken now, please don't approve anything | 11:14 |
| dtantsur | also unit tests are broken for IPA https://bugs.launchpad.net/ironic-python-agent/+bug/1522756 | 11:16 |
| openstack | Launchpad bug 1522756 in ironic-python-agent "Unit tests fail with DeviceNotFoundByFileError: [Errno 2] No such file or directory: '/dev/sda'" [High,Confirmed] - Assigned to Dmitry Tantsur (divius) | 11:16 |
| dtantsur | typical Friday, I would say | 11:16 |
| sambetts | yup :D everything breaks on Fridays its like even the machines know the weekend is coming | 11:19 |
| *** Nisha has quit IRC | 11:21 | |
| dtantsur | pyudev broke backward compatibility awesome | 11:22 |
| sambetts | yay! :D | 11:22 |
| vdrok | dtantsur, unittests in ironic too - https://review.openstack.org/#/c/253402/ | 11:23 |
| vdrok | morning everyone, tgif :D | 11:24 |
| lucasagomes | the security fix didn't get in yet | 11:27 |
| pas-ha | morning Ironic | 11:27 |
| dtantsur | morning pas-ha! join the celebration of broken gates | 11:28 |
| lucasagomes | dtantsur, yeah, oh friday | 11:28 |
| pas-ha | lucasagomes, I suspect this is related to problem being fixed in this patch https://review.openstack.org/#/c/253402/ | 11:29 |
| * lucasagomes looks | 11:30 | |
| lucasagomes | pas-ha, vdrok that's what causing the gate to break? | 11:32 |
| * lucasagomes check more logs | 11:32 | |
| pas-ha | well, the py27 error in cve patch is exactly in this testcase | 11:32 |
| openstackgerrit | Vladyslav Drok proposed openstack/ironic-specs: Add support for hardware composition https://review.openstack.org/253452 | 11:34 |
| pas-ha | concurrent unit tests run always bring interesting problems ^) | 11:34 |
| *** ndipanov has joined #openstack-ironic | 11:35 | |
| openstackgerrit | Dmitry Tantsur proposed openstack/ironic-python-agent: pyudev exception has changed for from_device_file https://review.openstack.org/253454 | 11:38 |
| dtantsur | discussing the issue on #openstack-stable too, if someone cares | 11:42 |
| *** chlong has joined #openstack-ironic | 11:44 | |
| *** sinval-afk is now known as sinval | 11:47 | |
| sinval | hey hey Ironicers, good morning | 11:47 |
| dtantsur | morning sinval | 11:47 |
| sinval | quick question, is 4.3 released? | 11:48 |
| dtantsur | I don't think so | 11:49 |
| sinval | dtantsur, thanks | 11:51 |
| lucasagomes | sinval, morning | 11:51 |
| sinval | lucasagomes, o/ | 11:52 |
| openstackgerrit | Merged openstack/ironic: Fix awake AMT unit test https://review.openstack.org/253402 | 11:58 |
| *** noqa_v_g1ovnie is now known as noqa_v_qoovnie | 11:58 | |
| * lucasagomes rechecks the security fix now | 12:03 | |
| *** yuanying_ has joined #openstack-ironic | 12:04 | |
| *** smoriya_ has quit IRC | 12:05 | |
| dtantsur | lucasagomes, please don't recheck/approve anything until gate is fixed | 12:05 |
| lucasagomes | dtantsur, it seems fixed | 12:05 |
| lucasagomes | ^ | 12:05 |
| lucasagomes | after the AMT stuff | 12:05 |
| dtantsur | lucasagomes, yeah, but tox jobs do nothing | 12:05 |
| lucasagomes | we gotta get this security fix in | 12:05 |
| lucasagomes | oh | 12:05 |
| dtantsur | see "[all][infra] All tox jobs broken, don't approve changes for now" | 12:05 |
| lucasagomes | fair enuff | 12:06 |
| dtantsur | I'd not do anything until a new notice from them | 12:06 |
| lucasagomes | ++ yeah sorry for that | 12:06 |
| sambetts | wow... can't they force them to -1 until they fix it to prevent accidental merges of borked code? | 12:06 |
| *** hanchao has quit IRC | 12:06 | |
| lucasagomes | tbh, I want that security fix to get in regardless | 12:06 |
| lucasagomes | it should have been merged yesterday | 12:06 |
| lucasagomes | since now it's open to the public | 12:07 |
| dtantsur | lucasagomes, well, maybe it's fair. I'm just making sure you're aware of the situation :) | 12:08 |
| lucasagomes | dtantsur, yeah thanks for that | 12:08 |
| *** yuanying has quit IRC | 12:08 | |
| *** dtantsur is now known as dtantsur|brb | 12:09 | |
| *** mattymo has left #openstack-ironic | 12:12 | |
| *** kozhukalov_ has quit IRC | 12:13 | |
| *** masterbound has joined #openstack-ironic | 12:15 | |
| *** mbound_ has quit IRC | 12:16 | |
| *** mbound_ has joined #openstack-ironic | 12:17 | |
| *** masterbound has quit IRC | 12:17 | |
| *** yuanying_ has quit IRC | 12:23 | |
| *** kozhukalov_ has joined #openstack-ironic | 12:26 | |
| *** yuanying has joined #openstack-ironic | 12:26 | |
| *** yuanying has quit IRC | 12:27 | |
| *** yanghy has joined #openstack-ironic | 12:30 | |
| yanghy | Hello, sorry to bother you guys...could somebody take this patch, it's been there for a long time and no progress https://review.openstack.org/#/c/250675/ | 12:35 |
| *** trown|outttypeww is now known as trown | 12:36 | |
| lucasagomes | yanghy, will take a look | 12:37 |
| yanghy | lucasagomes, thanks a lot! | 12:38 |
| lucasagomes | yanghy, there's a request there for unittests | 12:38 |
| liliars | good morning everyone | 12:41 |
| lucasagomes | liliars, morning | 12:42 |
| yanghy | lucasagomes, yeah, so you all think it's a blocker right? then I will take some time to look at this, because I'm not quite familiar with ironic code now...this bug was originally found in magnum through code review, and I happen to see that ironic code also has this problem... | 12:43 |
| sambetts | Morning lucasagomes | 12:43 |
| sambetts | morning liliars | 12:43 |
| lucasagomes | yanghy, not a blocker. It's ideal to have unittests tho. If you don't mind I can create the unittests for you and upload the code | 12:44 |
| lucasagomes | yanghy, at the moment there's a problem with tox so we are not approving patches until the infra fix it | 12:44 |
| yanghy | lucasagomes, that will be great!~ it takes time to look into the code though :) | 12:44 |
| lucasagomes | yanghy, cool | 12:44 |
| * lucasagomes does it while gate is broken anyway | 12:45 | |
| yanghy | thanks a lot lucasagomes ! | 12:45 |
| lucasagomes | yanghy, thank YOU for fixing it | 12:46 |
| lucasagomes | and reportng the problem | 12:46 |
| *** dtantsur|brb is now known as dtantsur | 12:46 | |
| liliars | lucasagomes, sambetts o/ | 12:46 |
| *** thrash has joined #openstack-ironic | 12:47 | |
| *** thrash has quit IRC | 12:47 | |
| *** thrash has joined #openstack-ironic | 12:47 | |
| liliars | oh I see we have a not-so-nice-broken-gate Friday :( | 12:47 |
| dtantsur | morning liliars, yes, we do :) | 12:48 |
| openstackgerrit | Pavlo Shchelokovskyy proposed openstack/ironic-inspector-specs: CMDB integration https://review.openstack.org/253054 | 12:48 |
| yanghy | lucasagomes, it's my pleasure :) | 12:51 |
| *** thrash has quit IRC | 12:55 | |
| lucasagomes | liliars, yeah :-( happens from time to time | 12:55 |
| lucasagomes | either friday or monday | 12:55 |
| *** mbound_ has quit IRC | 12:57 | |
| *** thrash has joined #openstack-ironic | 13:04 | |
| *** thrash has quit IRC | 13:04 | |
| *** thrash has joined #openstack-ironic | 13:04 | |
| *** yanghy has quit IRC | 13:04 | |
| *** mbound_ has joined #openstack-ironic | 13:05 | |
| *** dprince has joined #openstack-ironic | 13:07 | |
| liliars | morning dtantsur :) | 13:09 |
| liliars | lucasagomes, well now we have it on a Friday close to release :( | 13:10 |
| TheJulia | Good morning | 13:10 |
| openstackgerrit | Lucas Alvares Gomes proposed openstack/python-ironicclient: Fix exceptions.from_response() parameter https://review.openstack.org/250675 | 13:11 |
| lucasagomes | liliars, lol | 13:12 |
| liliars | morning TheJulia | 13:12 |
| *** thrash has quit IRC | 13:15 | |
| *** moorryan has joined #openstack-ironic | 13:19 | |
| *** thrash has joined #openstack-ironic | 13:19 | |
| *** thrash has quit IRC | 13:19 | |
| *** thrash has joined #openstack-ironic | 13:19 | |
| liliars | do we have a date for spec-freeze already? | 13:19 |
| lucasagomes | liliars, nah, I don't think we have a spec freeze | 13:20 |
| lucasagomes | since we have a diff release model | 13:20 |
| dtantsur | liliars, we never freeze specs. neither do we guarantee when a patch implementing a spec will land | 13:20 |
| dtantsur | treat specs repo as a backlog for project, not an attempt to plan releases | 13:21 |
| * lucasagomes needs to write a spec for the json fields | 13:23 | |
| lucasagomes | do it now | 13:23 |
| liliars | dtantsur, "neither do we guarantee when a patch implementing a spec will land" yeah that part I knew | 13:24 |
| liliars | dtantsur, lucasagomes oh well, deadlines help me organize, so I was hoping.. ;p but thanks guys! will plan differently | 13:24 |
| dtantsur | liliars, previously we had a deadline around M3, so you can use it for self-organizing :) | 13:24 |
| dtantsur | if you don't merge your spec by M3, chances are very low it will get to Mitaka | 13:25 |
| *** sambetts has quit IRC | 13:25 | |
| liliars | dtantsur, thanks, that's a direction :) will do! | 13:25 |
| *** sambetts has joined #openstack-ironic | 13:26 | |
| *** degorenko is now known as degorenko|afk | 13:26 | |
| *** mbound_ has quit IRC | 13:27 | |
| *** Piet has quit IRC | 13:36 | |
| *** mbound_ has joined #openstack-ironic | 13:38 | |
| *** david-lyle has quit IRC | 13:43 | |
| openstackgerrit | Naohiro Tamura proposed openstack/ironic: Add oslo_config.Opt support in Ironic config generator https://review.openstack.org/253498 | 13:46 |
| *** pas-ha has quit IRC | 13:48 | |
| *** pas-ha has joined #openstack-ironic | 13:49 | |
| *** liliars is now known as liliars-afk | 13:56 | |
| *** caiobo_ has quit IRC | 13:57 | |
| *** liliars-afk has quit IRC | 13:57 | |
| *** baoli_ has joined #openstack-ironic | 13:58 | |
| *** marcusrafael has quit IRC | 13:59 | |
| *** mbound_ has quit IRC | 13:59 | |
| *** degorenko|afk is now known as degorenko | 13:59 | |
| *** sergek has quit IRC | 14:00 | |
| jroll | mornin y'all | 14:00 |
| *** sinval has quit IRC | 14:00 | |
| dtantsur | morning jroll! see above, we have Friday gate fun | 14:01 |
| jroll | :| | 14:01 |
| jroll | so I see tox is fixed now yeah? | 14:01 |
| dtantsur | jroll, I don't see an official notice | 14:01 |
| jroll | and now it's just IPA unit tests broken? | 14:01 |
| *** sergek has joined #openstack-ironic | 14:01 | |
| *** mbound_ has joined #openstack-ironic | 14:01 | |
| dtantsur | jroll, I assume tox is broken until we get a confirmation from infra | 14:01 |
| dtantsur | jroll, and yes, IPA unit tests | 14:02 |
| jroll | dtantsur: oops, I thought it was a topic change and saw normal topic was there | 14:02 |
| jroll | WELP | 14:02 |
| *** links has quit IRC | 14:02 | |
| dtantsur | it's weird they didn't update topics.. | 14:03 |
| jroll | I guess I should just back away slowly then :) | 14:03 |
| openstackgerrit | Vladyslav Drok proposed openstack/ironic: Add proxy related parameters in agent driver https://review.openstack.org/235493 | 14:05 |
| *** zhenguo has joined #openstack-ironic | 14:13 | |
| lucasagomes | jroll, morning | 14:21 |
| *** [1]cdearborn has joined #openstack-ironic | 14:27 | |
| lucasagomes | jroll, around? For the JSON fields stuff | 14:27 |
| lucasagomes | jroll, do we expect all json fields to be indexable? I think that things like extra, driver_internal_info and so on they could remain as json | 14:28 |
| *** Marga_ has joined #openstack-ironic | 14:28 | |
| lucasagomes | anyone has opnions on that ^ ? | 14:28 |
| lucasagomes | (that's about the way the are stored in the db, in the API they will remain as JSON ofc) | 14:28 |
| dtantsur | I'd start with properties/capabilities, and solve other separately, if we ever feel like | 14:29 |
| lucasagomes | dtantsur, right, driver_info ? | 14:29 |
| lucasagomes | dtantsur, I want to have a separated spec for capabiltiies, I would do properties first and then separate capability into another spec and field in the API | 14:29 |
| dtantsur | we in ironic-inspector would love to see driver_info searchable, but I'm afraid it's only us.. | 14:29 |
| lucasagomes | dtantsur, that's a valueable use case | 14:30 |
| lucasagomes | if we are doing 1 field, doing another is not a big deal | 14:30 |
| lucasagomes | I will propose the spec to do properties and driver_info | 14:30 |
| *** absubram has joined #openstack-ironic | 14:31 | |
| *** alexpilotti has quit IRC | 14:32 | |
| lucasagomes | dtantsur, yeah well actually I will think about it after the call | 14:32 |
| dtantsur | cool! | 14:32 |
| NobodyCam | good morning Ironicers | 14:37 |
| NobodyCam | and | 14:37 |
| NobodyCam | TGIF!!! | 14:37 |
| *** absubram has quit IRC | 14:37 | |
| vdrok | morning TheJulia jroll and NobodyCam ! | 14:39 |
| NobodyCam | morning lucasagomes dtantsur jroll vdrok sambetts jlvillal :) | 14:39 |
| dtantsur | morning NobodyCam! | 14:40 |
| *** absubram has joined #openstack-ironic | 14:41 | |
| *** rkharya has joined #openstack-ironic | 14:41 | |
| NobodyCam | :) | 14:41 |
| dtantsur | rkharya, hi, so what's the problem? | 14:45 |
| rkharya | openstack-ironic-discoverd.service logs has this error - Dec 04 09:34:20 overcloud-compute-0.localdomain ironic-discoverd[16749]: ERROR:ironic_discoverd.utils:Could not find a node for attributes {'bmc_address': u'0.0.0.0', 'mac': [u'00:25:b5:77:00:2c']} | 14:46 |
| dtantsur | sambetts, is this IPMI address expected for UCS ^^? | 14:47 |
| rkharya | pxe_ucs driver does not rely on bmc address, it directly talks to UCSM via Python-ucsSDK for power driver functionality | 14:47 |
| dtantsur | rkharya, then you have to use port (bring it back, if you deleted it after our previous conversation), and make sure you start introspection by its CLI, not just by powering on the machine | 14:48 |
| rkharya | yeh..it did that as you mentioned. cleaned up and started afresh via 'openstack introspection start <node-id>' command | 14:49 |
| rkharya | before that did the node registration with all the details | 14:50 |
| dtantsur | rkharya, do you have a port for MAC 00:25:b5:77:00:2c? | 14:51 |
| rkharya | dtantsur, not yet, expecting it to get created once instrospection completes. because if i create it first, then i get that mac-address listed in iptables drop list as soon i execute 'ironic port-create <>' command | 14:53 |
| jroll | lucasagomes: yeah, I agree with dtantsur, start with properties and driver_info, and go from there | 14:53 |
| lucasagomes | jroll, fair enuff | 14:53 |
| *** dansmith is now known as superdan | 14:53 | |
| lucasagomes | thanks | 14:53 |
| rkharya | dtantur, that prevents bare metal node to get the dhcp address | 14:53 |
| dtantsur | rkharya, it will get whitelisted after you start the introspection. without both BMC address and MAC ironic-inspector wont' be able to lookup your node | 14:53 |
| *** alexpilotti has joined #openstack-ironic | 14:54 | |
| rkharya | dtantsur, ok let me try with this again | 14:54 |
| rkharya | dtantsur, one question though out of BMC and MAC will it work if only MAC is available? | 14:55 |
| *** alex_xu is now known as alexus | 14:56 | |
| dtantsur | rkharya, IIRC yes | 14:56 |
| * dtantsur double-checks | 14:56 | |
| dtantsur | yeah, should be fine | 14:57 |
| rkharya | dtantsur, ok thanks let me try with manual port creation and get back | 14:57 |
| *** alexpilotti has quit IRC | 14:58 | |
| openstackgerrit | Sergii Turivnyi proposed openstack/python-ironicclient: Add tests for testing node-delete command https://review.openstack.org/251458 | 15:02 |
| *** mbound_ has quit IRC | 15:10 | |
| *** Marga_ has quit IRC | 15:12 | |
| *** ir2ivps9 has quit IRC | 15:12 | |
| dtantsur | we got broken by pyudev, requirements repo is broken by neutron, and all this is broken by something in infra. could it be better? | 15:16 |
| * dtantsur rage quits and runs to the forest to live with owls | 15:16 | |
| openstackgerrit | Sergii Turivnyi proposed openstack/python-ironicclient: Add tests for testing node-delete command https://review.openstack.org/251458 | 15:16 |
| NobodyCam | dtantsur: NOOOOOOOOO!!!!! | 15:17 |
| *** Nisha has joined #openstack-ironic | 15:17 | |
| *** HimanshuGarg has quit IRC | 15:17 | |
| *** Marga_ has joined #openstack-ironic | 15:19 | |
| *** kozhukalov_ has quit IRC | 15:20 | |
| *** dprince has quit IRC | 15:21 | |
| *** gabriel has quit IRC | 15:22 | |
| *** Nisha_away has joined #openstack-ironic | 15:23 | |
| *** Nisha has quit IRC | 15:24 | |
| sambetts | dtantsur: -_- my irssi client made me post a random message in openstack-meeting3 by accident... | 15:26 |
| dtantsur | I hope it was not too offensive :) | 15:26 |
| sambetts | dtantsur: I thought the tripleo/OSP thing should have created the port in ironic manaually? | 15:26 |
| *** Marga_ has quit IRC | 15:27 | |
| sambetts | dtantsur: because it had the wrong channel selected but wasn't telling me my auto complete wasn't working so I thought you'd actually rage quit | 15:27 |
| dtantsur | sambetts, it should, but we recommended deleting it last time :) | 15:27 |
| *** degorenko has quit IRC | 15:27 | |
| sambetts | dtantsur: ah | 15:28 |
| sambetts | whoops haha | 15:28 |
| lucasagomes | sambetts, hehe now your message is registered forever :-) | 15:29 |
| sambetts | lucasagomes: yep... | 15:29 |
| sambetts | haha | 15:29 |
| *** mtanino has joined #openstack-ironic | 15:31 | |
| *** degorenko has joined #openstack-ironic | 15:31 | |
| sambetts | dtantsur: Did you see the response to your naming ML email? | 15:32 |
| *** Nisha_away has quit IRC | 15:32 | |
| dtantsur | sambetts, I see one response now | 15:32 |
| sambetts | :-P its good | 15:33 |
| *** Nisha_away has joined #openstack-ironic | 15:33 | |
| dtantsur | sambetts, by the way. I was told by the release team that we are no longer *required* to have milestone tracking in launchpad. I'm inclined to stop doing that and drop information about all milestones | 15:33 |
| dtantsur | only use launchpad to track bugs and blueprints | 15:33 |
| sambetts | dtantsur: because reno? | 15:34 |
| dtantsur | yep | 15:34 |
| dtantsur | that puts unnecessary burden on me now, so I'd avoid it | 15:35 |
| dtantsur | jroll, do you plan to continue using milestones in launchpad for ironic? | 15:35 |
| *** mbound_ has joined #openstack-ironic | 15:36 | |
| jroll | dtantsur: ehhhh, dunno, probably not since we aren't using it for bugs | 15:36 |
| sambetts | How do we track what we're targeting for a specfic release now? | 15:36 |
| sambetts | or is it just when we feel we've done enough | 15:36 |
| sambetts | ? | 15:36 |
| dtantsur | sambetts, do we really need to? or rather: did we *really* do it previously? | 15:36 |
| jroll | I was doing it for completed stuff and as a thing to indicate I'd like to finish a thing for a release | 15:37 |
| jroll | but yeah, that's where the 'dunno' comes in for me | 15:37 |
| openstackgerrit | Sergii Turivnyi proposed openstack/python-ironicclient: Tests for testing node-set-power-state command https://review.openstack.org/251997 | 15:38 |
| sambetts | the things we want to finish is the part I think that would be missing | 15:38 |
| sambetts | completed stuff gets a reno note so thats fine and tracked by that | 15:38 |
| sambetts | but the list of things we want to complete before we release isn't anywhere else is it? | 15:39 |
| openstackgerrit | Sergii Turivnyi proposed openstack/python-ironicclient: Add tests for testing node-delete command https://review.openstack.org/251458 | 15:40 |
| jroll | not really, no | 15:40 |
| jroll | but we don't do a very good job of that today | 15:41 |
| jroll | mostly because launchpad is terrible | 15:41 |
| lucasagomes | ++ | 15:41 |
| *** jcoufal has quit IRC | 15:41 | |
| * lucasagomes call's finished... will grab a quick lunch | 15:41 | |
| sambetts | true :/ | 15:41 |
| *** lucasagomes is now known as lucas-hungry | 15:41 | |
| dtantsur | jroll, sambetts, IMO ideally we should use the importance field in launchpad for priorities | 15:42 |
| dtantsur | e.g. "critical" means NOW, "high" means this release | 15:42 |
| jroll | mhm | 15:42 |
| sambetts | yeah, I think that would be nice, I think thats why neutron have ditched BPs now and use bugs for everything | 15:43 |
| sambetts | even for BP things they use a bug with a special tag on it | 15:43 |
| dtantsur | bugs + specs? sounds not so bad.. | 15:43 |
| dtantsur | we do it in Red Hat with our bugzilla | 15:44 |
| *** jcoufal has joined #openstack-ironic | 15:44 | |
| sambetts | I think it works much better than BPs do | 15:44 |
| jroll | I'm more wondering when phabricator will be available for use | 15:44 |
| jroll | if soon, I'd rather just wait for that | 15:44 |
| sambetts | thats does it everything bugs right? | 15:44 |
| dtantsur | jroll, the same time as previous "solve-all-the-problems" solution? :D how was it called? | 15:44 |
| *** yonglihe has joined #openstack-ironic | 15:44 | |
| jroll | heh, storyboard | 15:45 |
| jroll | but storyboard was "let's make our own", this is just deploying an existing thing | 15:45 |
| *** mgould has joined #openstack-ironic | 15:45 | |
| jroll | I do kind of like neutron's RFE thing | 15:46 |
| sambetts | neutron also do specless RFEs if its straight forward enough | 15:46 |
| dtantsur | me too. I never found LP blueprints convenient.. and now that we also have specs, I see little sense in them | 15:46 |
| dtantsur | fair enough, yeah | 15:46 |
| sambetts | its like levels of work, bug, RFE, RFE + Spec | 15:47 |
| dtantsur | also, we already have bugs in ironic that are essentially RFE's for small additions | 15:47 |
| dtantsur | should I bring it to the ML and then to the meeting? | 15:49 |
| jroll | dtantsur: sounds good, maybe the december 14 meeting to give folks time | 15:50 |
| *** dprince has joined #openstack-ironic | 15:50 | |
| * dtantsur writes yet another email | 15:50 | |
| *** jerrygb has joined #openstack-ironic | 15:52 | |
| *** alexpilotti has joined #openstack-ironic | 16:01 | |
| *** jerrygb_ has joined #openstack-ironic | 16:03 | |
| *** jerrygb has quit IRC | 16:06 | |
| *** jerrygb_ has quit IRC | 16:07 | |
| *** mgould has quit IRC | 16:09 | |
| *** jerrygb has joined #openstack-ironic | 16:10 | |
| openstackgerrit | Merged openstack/bifrost: Initial support for installation of ironic-inspector https://review.openstack.org/229930 | 16:18 |
| openstackgerrit | Merged openstack/bifrost: ironic-inspector install doc polish https://review.openstack.org/234810 | 16:18 |
| *** lucas-hungry is now known as lucasagomes | 16:19 | |
| *** mgould has joined #openstack-ironic | 16:22 | |
| *** mbound_ has quit IRC | 16:25 | |
| *** mbound_ has joined #openstack-ironic | 16:26 | |
| *** alexpilotti has quit IRC | 16:28 | |
| *** ifarkas has quit IRC | 16:28 | |
| dtantsur | FYI: tox is ok again, we can approve patches (to everything except for IPA) | 16:30 |
| dtantsur | lucasagomes, jroll ^^ | 16:30 |
| lucasagomes | dtantsur, w00t! | 16:31 |
| * lucasagomes recheck security fix | 16:31 | |
| jroll | thanks dtantsur | 16:31 |
| jroll | lol | 16:31 |
| *** alexpilotti has joined #openstack-ironic | 16:31 | |
| jroll | I just did | 16:31 |
| lucasagomes | oh | 16:31 |
| mariojv | hi - quick question about code review. 2 x +2s and 1 x +1 does not mean workflow +1 (and thus a merge), right? | 16:32 |
| lucasagomes | folks this had a +2 already but I added unittests (talked to the owner of the patch on IRC today) | 16:32 |
| lucasagomes | https://review.openstack.org/#/c/250675/ | 16:32 |
| lucasagomes | can we have some eyes on that? | 16:32 |
| lucasagomes | it's a simple patch | 16:32 |
| mariojv | never mind, checked contributor guidelines... :) i've got 2 +2s on https://review.openstack.org/#/c/248885/ (notifications) | 16:33 |
| *** mbound_ has quit IRC | 16:38 | |
| *** mbound_ has joined #openstack-ironic | 16:38 | |
| devananda | morning, all | 16:39 |
| *** mbound_ has quit IRC | 16:40 | |
| dtantsur | morning devananda | 16:40 |
| *** jerrygb_ has joined #openstack-ironic | 16:40 | |
| *** mbound_ has joined #openstack-ironic | 16:40 | |
| lucasagomes | devananda, morning | 16:41 |
| * dtantsur has sent a email about milestones and blueprints | 16:42 | |
| *** jerrygb has quit IRC | 16:42 | |
| openstackgerrit | Vladyslav Drok proposed openstack/ironic: Add documentation for proxies usage with IPA https://review.openstack.org/250878 | 16:43 |
| *** degorenko is now known as _degorenko|afk | 16:44 | |
| *** kozhukalov_ has joined #openstack-ironic | 16:47 | |
| *** david-lyle has joined #openstack-ironic | 16:49 | |
| *** alexpilotti has quit IRC | 16:52 | |
| *** rcernin has quit IRC | 16:52 | |
| dtantsur | leaving for a week now \o/ can someone please keep an eye on https://review.openstack.org/#/c/253448/ ? | 16:53 |
| -openstackstatus- NOTICE: The earlier JJB bug which disrupted tox-based job configurations has been reverted and applied; jobs seem to be running successfully for the past two hours. | 16:53 | |
| dtantsur | see you | 16:54 |
| sambetts | o/ dtantsur | 16:54 |
| JayF | o/ dtantsur | 16:54 |
| *** dtantsur is now known as dtantsur|afk | 16:54 | |
| *** Piet__ has joined #openstack-ironic | 16:55 | |
| mgould | o/ dtantsur|afk | 16:56 |
| *** mbound_ has quit IRC | 17:01 | |
| openstackgerrit | Lucas Alvares Gomes proposed openstack/ironic-specs: Add indexable JSON fields spec https://review.openstack.org/253605 | 17:03 |
| lucasagomes | devananda, jroll ^ it's a succinct spec, but hope the whole idea is there ^ | 17:03 |
| lucasagomes | dtantsur|afk, have a good weekend! | 17:07 |
| lucasagomes | se eya | 17:07 |
| mgould | hi everyone | 17:10 |
| mgould | I'm gradually unsticking myself on https://review.openstack.org/#/c/235158/ | 17:10 |
| mgould | but I have a question about the state machine | 17:10 |
| mgould | looking at http://docs.openstack.org/developer/ironic/_images/states.svg, it looks like a node can go enroll -> verifying -> enroll | 17:11 |
| mgould | which signals a failure | 17:11 |
| mgould | 1) dtantsur|afk's initial patch only makes sense if a node can go enroll -> manageable -> enroll on failure; can this ever happen? | 17:12 |
| mgould | 2) are we committed to this? It seems a very unreliable way to signal failure | 17:12 |
| mgould | check node: "enroll"; sleep; check node: "enroll" | 17:12 |
| *** david-lyle has quit IRC | 17:13 | |
| mgould | you don't know if it's failed or just not started transitioning yet | 17:13 |
| JayF | I believe in that cast | 17:13 |
| JayF | *case | 17:13 |
| JayF | node[last_error] would be populated | 17:13 |
| mgould | aha | 17:13 |
| JayF | For our internal dashboards, we usually key off a combo of state, target state, and last error | 17:13 |
| JayF | to suss out error situations | 17:13 |
| mgould | excellent | 17:14 |
| lucasagomes | yeah, the last_error should indicate what went wrong when a transition actually failed | 17:14 |
| devananda | JayF: as an aside, I've found several cases where last_error gets set by one process and cleared by another | 17:14 |
| lucasagomes | since when in enroll/manageable the node is not used yet I'm not sure if maintenace flag would make sense yet | 17:14 |
| JayF | devananda: usually the power status loop in my observation | 17:15 |
| devananda | we more and more need an event/error log // notifiation bus | 17:15 |
| devananda | JayF: yah | 17:15 |
| JayF | I kinda like the way Nova does it | 17:15 |
| JayF | with instance /errors/ | 17:15 |
| JayF | we could similarly have node errors in a separate table, perhaps | 17:15 |
| devananda | yup | 17:15 |
| mgould | OK, so my set_node_state function should check node[last_error], try to set the state, then loop until last_error changes or the node is in the target state? | 17:15 |
| devananda | client could ask for last error, or all errors since $ref | 17:16 |
| mgould | even better | 17:16 |
| mgould | wait, is that available now, or something you're proposing? | 17:16 |
| devananda | proposng | 17:17 |
| mgould | OK | 17:17 |
| JayF | mgould: node[last_error] always has the most recent node error | 17:17 |
| mgould | yes, that sounds useful :-) | 17:17 |
| JayF | mgould: devananda and I have always hated that we only keep the most recent error, so just daydreaming about fixing it :) | 17:17 |
| mgould | JayF, gotcha | 17:17 |
| mgould | is last_error timestamped? | 17:17 |
| *** cemason has joined #openstack-ironic | 17:17 | |
| *** rkharya has quit IRC | 17:17 | |
| *** alexpilotti has joined #openstack-ironic | 17:17 | |
| lucasagomes | JayF, ++ having the concept of instance | 17:18 |
| JayF | mgould: no | 17:18 |
| lucasagomes | claims is somewhat similar | 17:19 |
| mgould | I'm wondering if I can tell "this happened before I started, but the process completed fine" from "this error happened before, and it happened again when you tried it" | 17:19 |
| mgould | curses | 17:19 |
| * sambetts is calling it a day | 17:19 | |
| JayF | lucasagomes: well, I was thinking more a "node errors" table that maps to the idea of Nova's instance errors table | 17:19 |
| sambetts | night all o/ | 17:19 |
| JayF | mgould: you can clear last error as a client, iirc | 17:19 |
| mgould | sambetts, good night! | 17:19 |
| lucasagomes | JayF, ah right, yeah I was thinking differently | 17:19 |
| JayF | mgould: and for nodes in ENROLL state, not a whole lot (including power status loop) will be running on them yet | 17:19 |
| *** sambetts is now known as sambetts-afk | 17:19 | |
| mgould | JayF, OK, that'll work | 17:19 |
| mgould | JayF, good to know | 17:20 |
| JayF | np | 17:20 |
| mgould | but I also have to deal with available nodes | 17:20 |
| mgould | ...which apparently can't fail to transition to manageable, so that's OK | 17:20 |
| lucasagomes | I was thinking more about having /instance (which is similar to claim) e.g, create an instance with those properties (mapped to the nodes) and deploy it, so we can track the errors at /instance/<instance uuid> and so on | 17:20 |
| lucasagomes | it's a big change | 17:21 |
| lucasagomes | sambetts-afk, g'night! | 17:21 |
| mgould | lucasagomes, that sounds like a Good Thing, but I don't know how hard it would be | 17:21 |
| lucasagomes | yeah changes our API a lot | 17:21 |
| lucasagomes | so probably it's a no for now | 17:21 |
| lucasagomes | but v2 who knows we can better architect things | 17:21 |
| *** shakamunyi has quit IRC | 17:21 | |
| lucasagomes | now we have experience dealing with this, when we first created the API was hard because we didn't know exactly how everything would work | 17:22 |
| JayF | lucasagomes: I don't even care about having huge in-depth instance information | 17:23 |
| JayF | lucasagomes: just simply keeping a historical "all the instance uuids ever on this node" would make some operability things 100x easier | 17:23 |
| JayF | lucasagomes: you can ask Nova that question today; but only via db and it's obtuse | 17:24 |
| lucasagomes | right, yeah | 17:24 |
| lucasagomes | the instance_uuid field in ironic is very hmm "soft" idk how to say | 17:24 |
| lucasagomes | cause for the standalone case it doesn't even eed to be populated | 17:24 |
| lucasagomes | need* | 17:24 |
| lucasagomes | but yeah can be worked out | 17:25 |
| openstackgerrit | Lucas Alvares Gomes proposed openstack/ironic-specs: Add indexable JSON fields spec https://review.openstack.org/253605 | 17:25 |
| * lucasagomes fixes line's too long errors | 17:25 | |
| *** kozhukalov_ has quit IRC | 17:29 | |
| lucasagomes | devananda, hi there, when you get some free time mind taking a peek at https://review.openstack.org/#/c/252531 see if it's aligned with the suggestion you gave? | 17:31 |
| *** vishwanathj has quit IRC | 17:35 | |
| openstackgerrit | Lucas Alvares Gomes proposed openstack/ironic-specs: Add indexable JSON fields spec https://review.openstack.org/253605 | 17:38 |
| *** cemason has quit IRC | 17:38 | |
| NobodyCam | noight sambetts-afk | 17:38 |
| NobodyCam | nighgt dtantsur|afk | 17:39 |
| NobodyCam | night even | 17:39 |
| NobodyCam | morning devananda, JayF, mgould :) | 17:39 |
| JayF | gm | 17:40 |
| lucasagomes | NobodyCam, morning | 17:40 |
| NobodyCam | :) | 17:40 |
| mgould | NobodyCam, morning :-) | 17:41 |
| *** alexpilotti has quit IRC | 17:44 | |
| *** alexpilotti has joined #openstack-ironic | 17:44 | |
| NobodyCam | :) | 17:44 |
| *** derekh has quit IRC | 17:45 | |
| NobodyCam | am I remembering correctly that closes-bug tag reguires reno notes? | 17:46 |
| mgould | NobodyCam, I think so | 17:46 |
| *** alexpilotti has quit IRC | 17:48 | |
| *** cemason has joined #openstack-ironic | 17:49 | |
| openstackgerrit | Merged openstack/bifrost: Initial addition of an inspection module https://review.openstack.org/232745 | 17:50 |
| JayF | lucasagomes: not sure I grok what node_driver_info and node_properties have that are different from driver_info and properties (in your indexable fields spec) | 17:52 |
| *** cemason has quit IRC | 17:53 | |
| lucasagomes | JayF, driver_info and properties are today a column in the node's table. In that spec each one would be a separated table | 17:53 |
| lucasagomes | so one could query stuff like | 17:53 |
| JayF | I grok that part | 17:53 |
| JayF | but why replace the two fields with 4 tables? | 17:53 |
| lucasagomes | SELECT node_id WHERE property_id=(SELECT id FROM properties WHERE field_name='memory_mb') AND value='1024' | 17:54 |
| lucasagomes | JayF, so we can query on it | 17:55 |
| lucasagomes | where having a blob makes it impossible | 17:55 |
| lucasagomes | well not impossible, but inefficient | 17:55 |
| JayF | OH, because we need one big flat table for all nodes | 17:56 |
| JayF | so you have to index what maps to what | 17:56 |
| lucasagomes | yes | 17:56 |
| JayF | Took me a second to understand, thanks for spelling it out | 17:56 |
| JayF | I've interacted with a lot of API + SQL schemas, just now learning how to build them. That makes a lot of sense, thanks | 17:57 |
| lucasagomes | JayF, that spec is just plumbing work. After that we can go ahead and put an API on top of it | 17:57 |
| lucasagomes | which then can be used for claims as well | 17:57 |
| lucasagomes | JayF, no problem | 17:57 |
| lucasagomes | but yeah keep in mind that, that spec alone doesn't bring any apparent benefit. Because it's all internal stuff | 17:58 |
| mgould | so each node can have many properties; each property has a name and a JSON value; most property names are common to many nodes? | 17:58 |
| lucasagomes | mgould, exactly | 17:58 |
| mgould | cool | 17:58 |
| lucasagomes | mgould, pretty much all nodes have cpus, memory_mb, local_gb, capabilities and so on | 17:58 |
| lucasagomes | it's duplicated across all nodes, so we only need to register it once | 17:59 |
| mgould | and currently we have a "properties" field in the nodes table, which is a string containing JSON? | 17:59 |
| lucasagomes | mgould, exactly | 17:59 |
| lucasagomes | it's a JSON blob stored in the node's table | 17:59 |
| lucasagomes | nodes* | 17:59 |
| mgould | right | 17:59 |
| mgould | so an alternative would be to make that field a JSON field rather than a VARCHAR, and use MySQL's JSON-indexing features | 18:00 |
| lucasagomes | mgould, yeah well the value has to be json because we currently support things like nested dicts in those fields | 18:01 |
| lucasagomes | cause they are json fields | 18:01 |
| mgould | sure, yes | 18:01 |
| lucasagomes | so, I want to keep the same | 18:01 |
| lucasagomes | oslo.db does have a JSONType that I will use for that | 18:01 |
| JayF | MySQL has a native JSON field type now? | 18:01 |
| JayF | TIL; I thought only Postgres did | 18:01 |
| mgould | JayF, apparently so: https://dev.mysql.com/doc/refman/5.7/en/json.html | 18:01 |
| lucasagomes | https://github.com/openstack/oslo.db/blob/master/oslo_db/sqlalchemy/types.py#L18 | 18:01 |
| mgould | aaargh | 18:01 |
| mgould | wait, sorry, my link's fine | 18:02 |
| mgould | lucasagomes, I'm talking about the node table having a native-JSON field containing eg {'cpus': 3, 'capabilities': ['things', 'stuff', 'more things']} | 18:03 |
| mgould | ie, what we have now but using MySQL's native JSON type rather than VARCHAR | 18:03 |
| lucasagomes | oh I see | 18:03 |
| lucasagomes | so we wouldn't need to dumps() loads() | 18:03 |
| mgould | yeah, I think so | 18:04 |
| mgould | and searching can be fast because the DB holds it all in an indexed form | 18:04 |
| lucasagomes | right so we wouldn't even need to change our db model | 18:04 |
| mgould | NB I have never actually used this feature :-) | 18:04 |
| mgould | what version of MySQL do we require? | 18:04 |
| lucasagomes | we use an oRM | 18:04 |
| mgould | lucasagomes, right | 18:04 |
| lucasagomes | ORM* | 18:05 |
| *** praneshp has joined #openstack-ironic | 18:05 | |
| mgould | we'd need to migrate the field type | 18:05 |
| lucasagomes | so we abstract the backend | 18:05 |
| lucasagomes | mgould, yeah | 18:05 |
| mgould | ah, OK | 18:05 |
| *** raildo has joined #openstack-ironic | 18:05 | |
| lucasagomes | mgould, that's the problem for us now | 18:05 |
| mgould | right | 18:05 |
| mgould | so if the ORM doesn't support native JSON types, my plan won't work | 18:05 |
| lucasagomes | yeah | 18:05 |
| raildo | pas-ha: ping, are you around? | 18:06 |
| lucasagomes | mgould, I will look into it see if I can find something | 18:06 |
| mgould | lucasagomes, cool | 18:06 |
| *** Piet__ has quit IRC | 18:06 | |
| lucasagomes | but I don't think it has something like that because we use things like sqllite for tests and so on | 18:07 |
| mgould | ah, right | 18:07 |
| lucasagomes | unittests* | 18:07 |
| mgould | I'll leave a comment on your patch saying "this could be an alternative if the ORM supports it" - would that make sense? | 18:07 |
| lucasagomes | mgould, totally | 18:07 |
| mgould | hang on, you're specifying the type of field_value as JSON | 18:08 |
| mgould | will that work if the ORM doesn't support it? | 18:08 |
| *** jcoufal has quit IRC | 18:08 | |
| *** cemason has joined #openstack-ironic | 18:08 | |
| mgould | I suppose it might support fetch and update of JSON types, but not querying | 18:08 |
| lucasagomes | mgould, this is the oslo.db type | 18:09 |
| lucasagomes | from https://github.com/openstack/oslo.db/blob/master/oslo_db/sqlalchemy/types.py#L18 | 18:09 |
| lucasagomes | but that dumps() and loads() | 18:09 |
| *** aginwala has joined #openstack-ironic | 18:09 | |
| lucasagomes | mgould, that's wrong in the spec actually, it should be TEXT | 18:09 |
| * lucasagomes changes it | 18:09 | |
| mgould | dammit | 18:10 |
| mgould | oh well | 18:10 |
| lucasagomes | >.< | 18:10 |
| lucasagomes | sorry | 18:10 |
| lucasagomes | hah | 18:10 |
| mgould | PSA: native JSON types exist in both MySQL and Postgres now :-) | 18:10 |
| lucasagomes | http://docs.sqlalchemy.org/en/latest/core/type_basics.html#generic-types | 18:10 |
| * lucasagomes will wait for mgould comment so he can add the alternative thing | 18:11 | |
| mgould | done | 18:11 |
| lucasagomes | thanks a lot | 18:12 |
| mgould | yeah, looks like JSON hasn't made it into SQLAlchemy's MySQL backend yet | 18:13 |
| mgould | though the Postgres backend supports Postgres' JSON type :-) | 18:14 |
| *** aginwala has quit IRC | 18:14 | |
| lucasagomes | yeah | 18:14 |
| mgould | right, I have to go to the first ever Red Hat Scotland christmas party :-) | 18:15 |
| mgould | at which I will meet most of the Scottish Red Hatters for the first time | 18:15 |
| mgould | have a good weekend, everyone! | 18:15 |
| *** rajathagasthya has joined #openstack-ironic | 18:16 | |
| lucasagomes | mgould, thanks for the suggestions/comments! Enjoy the weekend | 18:17 |
| lucasagomes | enjoy the party as well | 18:17 |
| mgould | thanks! | 18:17 |
| *** mgould has quit IRC | 18:18 | |
| openstackgerrit | Lucas Alvares Gomes proposed openstack/ironic-specs: Add indexable JSON fields spec https://review.openstack.org/253605 | 18:23 |
| *** trown is now known as trown|lunch | 18:24 | |
| openstackgerrit | Julia Kreger proposed openstack/bifrost: WIP: Canary test commit **DO NOT MERGE** https://review.openstack.org/253639 | 18:25 |
| *** kkoski has quit IRC | 18:28 | |
| *** vishwanathj has joined #openstack-ironic | 18:32 | |
| *** Marga_ has joined #openstack-ironic | 18:33 | |
| *** Marga_ has quit IRC | 18:34 | |
| *** Marga_ has joined #openstack-ironic | 18:35 | |
| *** vishwanathj has quit IRC | 18:36 | |
| *** penick has joined #openstack-ironic | 18:37 | |
| *** jaybeale has joined #openstack-ironic | 18:38 | |
| *** cemason has quit IRC | 18:40 | |
| *** kozhukalov_ has joined #openstack-ironic | 18:41 | |
| *** aginwala has joined #openstack-ironic | 18:46 | |
| *** kkoski has joined #openstack-ironic | 18:46 | |
| *** Nisha_brb has joined #openstack-ironic | 18:48 | |
| *** Nisha_away has quit IRC | 18:48 | |
| openstackgerrit | Merged openstack/ironic: Fix bug where clean steps do not run https://review.openstack.org/252993 | 18:49 |
| JayF | finally \o/ | 18:50 |
| *** aginwala has quit IRC | 18:52 | |
| *** ChubYann has joined #openstack-ironic | 18:53 | |
| *** aginwala has joined #openstack-ironic | 18:54 | |
| *** aginwala has quit IRC | 18:58 | |
| *** electrofelix has quit IRC | 19:00 | |
| *** aginwala has joined #openstack-ironic | 19:01 | |
| *** penick has quit IRC | 19:05 | |
| lucasagomes | \o/ | 19:05 |
| *** david-ly_ has joined #openstack-ironic | 19:06 | |
| *** HimanshuGarg has joined #openstack-ironic | 19:08 | |
| *** pas-ha has quit IRC | 19:12 | |
| devananda | woot! | 19:13 |
| *** HimanshuGarg has quit IRC | 19:13 | |
| devananda | time to tag that release, eh? jroll - you around, or shall I do it? | 19:13 |
| *** trown|lunch is now known as trown | 19:14 | |
| *** jerrygb has joined #openstack-ironic | 19:16 | |
| *** jerrygb_ has quit IRC | 19:18 | |
| jroll | devananda: go for it, I'm about to hit the road. thank you! | 19:28 |
| jroll | don't forget liberty :P | 19:28 |
| *** penick has joined #openstack-ironic | 19:29 | |
| devananda | jroll: have a great weekend :) | 19:29 |
| jroll | thanks, you too! | 19:30 |
| * jroll off to STL | 19:30 | |
| *** Sukhdev has joined #openstack-ironic | 19:33 | |
| *** HimanshuGarg has joined #openstack-ironic | 19:33 | |
| lucasagomes | jroll, enjoy! | 19:35 |
| NobodyCam | have a good weekend jroll | 19:39 |
| *** david-ly_ is now known as david-lyle | 19:39 | |
| *** rajathagasthya has quit IRC | 19:41 | |
| Nisha_brb | devananda, i was testing manual cleaning through curl | 19:41 |
| NobodyCam | mgoddard: around? wanted to check if your still working on https://review.openstack.org/#/c/153230 ??? | 19:42 |
| Nisha_brb | i am facing an issue, could you/anyone else tell me what am i doing wrong | 19:43 |
| Nisha_brb | Here is the paste | 19:43 |
| Nisha_brb | http://paste.openstack.org/show/480932/ | 19:43 |
| Nisha_brb | NobodyCam, ^^^ | 19:43 |
| openstackgerrit | John L. Villalovos proposed openstack/ironic: Documentation points to official release notes https://review.openstack.org/250529 | 19:43 |
| *** rajathagasthya has joined #openstack-ironic | 19:49 | |
| *** jaybeale has quit IRC | 19:50 | |
| *** mgarza_ has joined #openstack-ironic | 19:51 | |
| openstackgerrit | Lucas Alvares Gomes proposed openstack/ironic-inspector-specs: test https://review.openstack.org/253674 | 19:52 |
| *** Marga_ has quit IRC | 19:53 | |
| *** mgarza_ has quit IRC | 19:53 | |
| *** rajathagasthya has quit IRC | 19:53 | |
| *** mjturek1 has quit IRC | 19:53 | |
| openstackgerrit | Milan Kováčik proposed openstack/ironic-inspector-specs: High Availability for Ironic Inspector https://review.openstack.org/253675 | 19:55 |
| raildo | pas-ha: ping, are you around? it's about two bugs that you're working :) #1494776 and #1422632 | 19:59 |
| *** ChuckC has joined #openstack-ironic | 20:05 | |
| NobodyCam | Nisha_brb: maybe https://github.com/openstack/ironic/blob/master/ironic/api/controllers/v1/node.py#L348-L349 | 20:06 |
| lucasagomes | folks calling it a day | 20:08 |
| lucasagomes | have a great night everyone! Enjoy tehe weekend | 20:08 |
| NobodyCam | lucasagomes: have a great weekend :) | 20:08 |
| *** baoli_ has quit IRC | 20:09 | |
| lucasagomes | NobodyCam, you too! See ya | 20:10 |
| *** lucasagomes is now known as lucas-afk | 20:10 | |
| *** mkovacik_ has quit IRC | 20:10 | |
| *** Nisha_brb has quit IRC | 20:12 | |
| *** shakamunyi has joined #openstack-ironic | 20:13 | |
| *** aginwala has quit IRC | 20:13 | |
| *** aginwala has joined #openstack-ironic | 20:17 | |
| *** mgoddard has quit IRC | 20:25 | |
| *** mgoddard has joined #openstack-ironic | 20:26 | |
| *** alexpilotti has joined #openstack-ironic | 20:26 | |
| mgoddard | NobodyCam: I'm around briefly | 20:27 |
| NobodyCam | hey hey mgoddard | 20:27 |
| mgoddard | hey | 20:28 |
| NobodyCam | I was looking at https://review.openstack.org/#/c/153230 | 20:28 |
| mgoddard | I'm not actively working on that bug | 20:28 |
| *** aginwala has quit IRC | 20:28 | |
| NobodyCam | but you implatmented a solution based on that for your own devices? | 20:28 |
| mgoddard | I proposed a solution, and was hoping for feedback but didn't get much | 20:28 |
| mgoddard | yes we're using a solution based on the same idea | 20:29 |
| NobodyCam | ya was reading theu the bug | 20:29 |
| mgoddard | would be great if the thread could be awoken - it seems like a fairly big limitation of Ironic IMHO | 20:30 |
| *** aginwala has joined #openstack-ironic | 20:31 | |
| mgoddard | if you're willing and able to nudge Nova reviews along then would appreciate a vote (for or against) on https://review.openstack.org/#/c/230919/ | 20:32 |
| *** vishwanathj has joined #openstack-ironic | 20:33 | |
| *** alexpilotti has quit IRC | 20:33 | |
| NobodyCam | not sure if I can influnce nova review but will take a look in a few | 20:36 |
| mgoddard | great, thanks | 20:36 |
| *** Marga_ has joined #openstack-ironic | 20:37 | |
| *** vishwanathj has quit IRC | 20:38 | |
| *** alexpilotti has joined #openstack-ironic | 20:40 | |
| *** kozhukalov_ has quit IRC | 20:43 | |
| *** cemason has joined #openstack-ironic | 20:48 | |
| *** cemason has quit IRC | 20:53 | |
| *** alexpilotti has quit IRC | 20:53 | |
| *** raildo is now known as raildo-afk | 20:54 | |
| *** alexpilotti has joined #openstack-ironic | 20:55 | |
| *** aginwala has quit IRC | 20:57 | |
| JayF | #success Cleaning works in Ironic again. | 20:58 |
| openstackstatus | JayF: Added success to Success page | 20:58 |
| *** ionutbalutoiu has quit IRC | 20:59 | |
| *** ionutbalutoiu has joined #openstack-ironic | 20:59 | |
| *** rajathagasthya has joined #openstack-ironic | 20:59 | |
| *** penick has quit IRC | 21:01 | |
| *** mjturek1 has joined #openstack-ironic | 21:02 | |
| *** mbound_ has joined #openstack-ironic | 21:03 | |
| *** Sukhdev has quit IRC | 21:04 | |
| *** penick has joined #openstack-ironic | 21:06 | |
| NobodyCam | oh nice! | 21:07 |
| *** baoli has joined #openstack-ironic | 21:07 | |
| *** alexpilotti has quit IRC | 21:09 | |
| *** aginwala has joined #openstack-ironic | 21:14 | |
| *** vishwanathj has joined #openstack-ironic | 21:20 | |
| *** boris-42_ has joined #openstack-ironic | 21:27 | |
| *** alexpilotti has joined #openstack-ironic | 21:30 | |
| JayF | devananda: if you get a chance, mariojv's notifications spec has two +2s on it (one of them mine). You had comments on it earlier so wanted to give you a shot to review it again before it landed (my hope is you'll make it 3x+2 and land it) | 21:32 |
| mariojv | thanks JayF. here's a link to the spec: https://review.openstack.org/#/c/248885/ | 21:33 |
| devananda | JayF: awesome | 21:34 |
| devananda | mariojv: JayF: oh, good point about shipping the Node obj around. it can potentially be LARGE | 21:38 |
| devananda | also contain sensitive info | 21:38 |
| JayF | we should probably strip instance_info as well as any creds | 21:38 |
| devananda | eg, it can contain a base64 configdrive | 21:38 |
| JayF | Heh, yep | 21:38 |
| JayF | so it should be sanitized some | 21:38 |
| devananda | strip instance_info, driver_info | 21:38 |
| devananda | actually, why not just pass the node UUID? | 21:39 |
| devananda | don't assume that the event receiver has privileges sufficient to access Ironic | 21:39 |
| mariojv | we might want to retain some information about the node in the notification, like its state and target state at the time the notification had to be sent | 21:39 |
| devananda | mariojv: for an event related to a state transition - yes, send that information | 21:39 |
| devananda | for an event related to node creation / deletion, dont send that information | 21:40 |
| mariojv | sure | 21:40 |
| mariojv | here's an example of what nova sends if anyone's curious: https://wiki.openstack.org/wiki/SystemUsageData#compute.instance.create..7Bstart.2Cerror.2Cend.7D: | 21:40 |
| JayF | mariojv: it just occurred to me why you picked up the notifications piece -- you worked with nova notifications a lot in your pre-onmetal project at Rackspace, didn't you? | 21:42 |
| mariojv | yes, mostly from a consumer standpoint of using it in the context of metrics | 21:43 |
| JayF | that probably means you have a good perspective on what those notifications should include | 21:43 |
| JayF | I mean, it sounds like devananda is arguing it shouldn't be "one size fits all" (i.e. a node object with a few things stripped) but maybe sending along the relevant fields instead? | 21:44 |
| * JayF not 100% sure he followed but that's what I thought was meant | 21:44 | |
| NobodyCam | devananda: just wanted to check and see if you've put any more thought in to https://review.openstack.org/#/c/153230 | 21:44 |
| mariojv | well, i'd like to send just the relevant fields, but that's not always trivial to figure out for something like a .error event | 21:44 |
| JayF | Error might be the case where you ship as much stuff as possible :) | 21:45 |
| JayF | (implied: barring sensitive info) | 21:45 |
| mariojv | JayF: for example, downstream we have the idea of "moods" where there are known combinations of things that are symptoms of problems in our environment | 21:45 |
| JayF | mariojv: fwiw a versino of that dash was open sourced for one of JoshNang's talks (in Vancouver) | 21:46 |
| mariojv | yeah, couldn't find the link | 21:46 |
| JayF | mariojv: I'd suspect, though, we'd want those notifications to have all the info we'd use in a dash like that to key off "what happened" | 21:46 |
| mariojv | sometimes i use downstream to mean non-openstack | 21:46 |
| JayF | but that's generally: 1) What flavor is it? 2) What state is it in and targetting? 3) Is it locked? 4) What does last_error say? 5) Is it maint'd | 21:47 |
| JayF | mariojv: https://github.com/rackerlabs/onmetal-dashboard | 21:48 |
| mariojv | sometimes power state, instance uuid, time since last heartbeat as well - but yes they're finite | 21:48 |
| JayF | I mean, I think there are two key pieces of info, right? | 21:49 |
| mariojv | generally for each mood | 21:49 |
| mariojv | yes | 21:49 |
| *** aginwala has quit IRC | 21:49 | |
| *** alexpilotti has quit IRC | 21:49 | |
| mariojv | in either case i don't want to presume that the fields we're interested in are the ones that everyone is interested in | 21:49 |
| JayF | Stuff about the node itself (power state, node.properies) | 21:49 |
| devananda | JayF: yes, your understanding my my point is correct | 21:49 |
| JayF | and stuff about what was happening (states, requested transition, etc) | 21:50 |
| mariojv | so i'm more prone to the idea that we should ship more fields rather than less, but we shouldn't send fields that don't have any limit on their size | 21:50 |
| *** mgoddard has quit IRC | 21:50 | |
| JayF | I think that's a pretty reasonable course of action, actually | 21:50 |
| devananda | sending data which is relates to the node, but not the event notificatoin itself, is not helpful -- it might be stale by the time it is received and processed | 21:50 |
| JayF | devananda: I throughly disagree | 21:51 |
| devananda | we should only send data related to the event | 21:51 |
| devananda | JayF: great! why? | 21:51 |
| JayF | devananda: in troubleshooting strange stuff, I've oft wished I had a snapshot of what the node itself looked like at the time | 21:51 |
| devananda | ah | 21:51 |
| JayF | devananda: I guess the question is; do we expect the notification into to be used for troubleshooting? If so, I want as much info as possible, because the point-in-time snapshot is a very useful input to that troubelshooting | 21:51 |
| mariojv | we use notifications as a data store for systems that help us "debug" our cloud | 21:51 |
| mariojv | s/store/source | 21:51 |
| JayF | I mean, we just stuff notifications into elasticsearch and query them in fun ways | 21:52 |
| JayF | for nova | 21:52 |
| mariojv | we also do other things with them | 21:52 |
| JayF | mariojv: oh, I'm sure you know of a million more than I do :) | 21:52 |
| mariojv | for performance monitoring, for example | 21:52 |
| mariojv | :) | 21:52 |
| devananda | JayF: yea, that's fair. if you're using notifications as a PIT snapshot so you can reconstruct the changes a node went through | 21:52 |
| JayF | devananda: that's /exactly/ what I had in mind when saying that | 21:53 |
| devananda | then it makes sense to put as much data, even sensitive data, as possible | 21:53 |
| devananda | what if the password changed and that caused the BMC to be unreachable and trigger maintenance mode? | 21:53 |
| JayF | I mean, IDK about sensitive data though? | 21:53 |
| devananda | sanitizing the payload would prevent debugging that | 21:53 |
| devananda | it's terrible from a security POV | 21:53 |
| JayF | devananda: you have no idea how close to a bullseye on a recent problem we had you are :) | 21:53 |
| *** aginwala has joined #openstack-ironic | 21:53 | |
| devananda | ;) | 21:53 |
| JayF | I mean, it's a continuum, right? | 21:54 |
| JayF | Most of the data is not-sensitive and helpful, so including it seems sensible | 21:54 |
| mariojv | yeah. i see this going towards a more per-use-case sort of configurability | 21:54 |
| devananda | back to the premise -- what is the notification bus for? | 21:54 |
| mariojv | this is something we said we wanted to avoid at the summit, and have it standardized | 21:54 |
| *** vishwana_ has joined #openstack-ironic | 21:54 | |
| devananda | a) debugging / recreating a series of events | 21:54 |
| devananda | b) informing other services of specific changes in state, to enable those services to take an action | 21:54 |
| mariojv | b) sending notifications to inform external services of when ironic ... that ^ | 21:54 |
| devananda | yes | 21:54 |
| devananda | so | 21:54 |
| JayF | Why can't we have both? | 21:55 |
| devananda | my point above -- we should not be sending the whole Node object for (b) | 21:55 |
| devananda | we should only send the relevant portions for that notification | 21:55 |
| *** Sukhdev has joined #openstack-ironic | 21:55 | |
| devananda | because the service which acts upon that notification probably needs to know the CURRENT state of hte node, when it processes that notification | 21:55 |
| devananda | "hey, this node was added, and it had 0GB of RAM at enrollment" --> "the node now has 64GB of RAM" | 21:56 |
| JayF | What, other than a debugging tool, might you want to act on these notifications? | 21:56 |
| devananda | "this node failed to transition to power_off" --> the node is currently off, so don't do anything | 21:56 |
| devananda | JayF: uh, Nova and Neutron | 21:56 |
| mariojv | i'm kind of interested in that | 21:56 |
| JayF | devananda: OH! | 21:56 |
| JayF | devananda: So no more polling? | 21:56 |
| mariojv | will we assume the message bus is lossy? | 21:56 |
| devananda | JayF: another inventory management system | 21:56 |
| devananda | JayF: exactly!! | 21:56 |
| devananda | mariojv: yes | 21:56 |
| *** vishwanathj has quit IRC | 21:56 | |
| devananda | [CMDB] receives notification that Node N provision_state transition to active has started | 21:57 |
| devananda | [CMDB] quickly changes local property "managed: False", then starts up a watchdog to keep an eye on the server | 21:58 |
| devananda | things like that | 21:58 |
| JoshNang | mariojv: i think if you're going to have nova/neutron actions depending on bus notifications, you're going to need to ensure it isn't lossy | 21:58 |
| devananda | JoshNang: o/ | 21:59 |
| mariojv | i understand why some of the fields would be unnecessary for b), but i don't understand how being more verbose would prevent the notification bus from being consumed by nova or neutron | 21:59 |
| JayF | JoshNang: ++ I was thinking the same | 21:59 |
| mariojv | JoshNang: hi! | 21:59 |
| JayF | and if we can ensure, for instance, API->Conductor bus messages aren't lossy | 21:59 |
| JoshNang | pretty sure most message busses will support "ensure at least once"..."ensure exactly once" isn't doable iirc | 21:59 |
| JoshNang | o/ | 21:59 |
| JayF | why can't we do the same for notifications? | 21:59 |
| devananda | JoshNang: we'll still need to poll periodically to catch anything that fell through the cracks | 21:59 |
| devananda | JayF: we can't ensure that | 21:59 |
| mariojv | devananda: ++ | 21:59 |
| JayF | devananda: How do we handle a lost message between an API and a Conductor? | 22:00 |
| NobodyCam | hey JoshNang welcome back :) | 22:00 |
| devananda | JayF: we currently abuse rabbit horribly in Ironic | 22:00 |
| JoshNang | NobodyCam: o/ :D | 22:00 |
| JoshNang | saw a mention and found the conversation interesting | 22:00 |
| devananda | JayF: right now, we're using rabbit for RPC -- not as a message bus -- and so we send an ACK for every message | 22:00 |
| mariojv | FWIW, oslo.messaging supports multiple drivers | 22:00 |
| devananda | if that ACK isn't receivedin X time, we raise an error | 22:00 |
| JayF | devananda: Ahh, of course, that makes sense | 22:01 |
| devananda | JayF: we are not using it as a queue at all | 22:01 |
| devananda | but for notifications, we need to | 22:01 |
| devananda | we won't (and can't) know who all the receivers are | 22:01 |
| mariojv | not a lot though: http://docs.openstack.org/developer/oslo.messaging/drivers.html | 22:01 |
| JayF | Understood | 22:02 |
| devananda | now, there are durable lossless message queues out there | 22:02 |
| *** penick has quit IRC | 22:02 | |
| devananda | but scaling those to run at the volume of messages openstack passes around? yea, fun. | 22:02 |
| *** trown is now known as trown|outttypeww | 22:02 | |
| devananda | it would need to be a different message bus than the rabbit bus we use between services | 22:03 |
| mariojv | oslo messaging provides a separate notifications endpoint config option (separate from rpc) | 22:05 |
| *** penick has joined #openstack-ironic | 22:06 | |
| JayF | mariojv: so it sounds like maybe we should talk more specifically about the use case(s) in the spec and change the payload accordingly | 22:09 |
| mariojv | i really think that is something that should be hashed out in the implementation | 22:10 |
| mariojv | worried about scope creep where we define each notification payload for everything we'd want in the future in the spec | 22:10 |
| JayF | I maybe agree with you w/r/t that | 22:10 |
| JayF | but the use cases need to be explicitly spelled out | 22:11 |
| mariojv | sure | 22:11 |
| *** thrash is now known as thrash|wknd | 22:11 | |
| JayF | because there may be an explicit decision, based on some of the chatter in here, that we could reject the "debug" use case | 22:11 |
| mariojv | i think problem description talks about both, but it could have more detail on our specific case | 22:11 |
| JayF | I mean, either way, they are clearly two cases that call for radically different contents of the notification | 22:12 |
| JayF | unless we want to "pile on" the whole node obj minus a few things | 22:12 |
| JayF | which the more I think of it seems like the type of thing we may regret in a couple of years as the node object may grow | 22:12 |
| mariojv | i agree. that's why they're versioned, so we can deprecate things :D | 22:14 |
| mariojv | (kind of kidding, but we're avoiding a huge problem nova had for a while) | 22:14 |
| JayF | (in 2 years) <mariojv> I have to deprecate things D: | 22:15 |
| JayF | lol | 22:15 |
| mariojv | :) | 22:15 |
| JayF | what's the huge problem nova had for a while | 22:15 |
| *** jerrygb has quit IRC | 22:15 | |
| mariojv | not having versioned or standardized notifications | 22:15 |
| mariojv | i don't know of specific cases where that broke people, but it was very annoying | 22:15 |
| mariojv | JayF: https://github.com/openstack/nova-specs/blob/master/specs/ | 22:16 |
| mariojv | mitaka/approved/versioned-notification-api.rst | 22:16 |
| mariojv | sorry about link split | 22:16 |
| JayF | gotcha | 22:16 |
| *** aginwala_ has joined #openstack-ironic | 22:20 | |
| *** aginwala_ has quit IRC | 22:21 | |
| *** aginwala has quit IRC | 22:22 | |
| *** aginwala_ has joined #openstack-ironic | 22:22 | |
| mariojv | so, let me double check it's clear what should be added to the spec, and maybe what should be left for later | 22:22 |
| mariojv | 1) definitely add more info about the two uses cases we've been discussing, since people who read the spec weren't clear on it, it needs more detail | 22:23 |
| mariojv | 2) for the details of how use cases are isolated from each other, either a) add a config option for verbosity, or b) leave that for implementation | 22:23 |
| mariojv | i prefer 2b since more use cases may come out that might not fit with 1 config option | 22:24 |
| JayF | I suspect we aren't going to get the spec approved without figuring out the verbosity ahead of time, but we'll see :) | 22:24 |
| mariojv | devananda: JayF: does that cover it, or is there more? ^ | 22:24 |
| JayF | mariojv: I'd also suggest putting this question on the agenda for the ironic meeting business-tomorrow | 22:24 |
| mariojv | ++ | 22:24 |
| mariojv | JayF: is that something i need to request to have added as a topic myself, or something for open discussion? | 22:26 |
| mariojv | i see on the wiki that we need 2 days in advance to have a new topic, not sure if that's something i just request in irc or if there's a form or something | 22:27 |
| JayF | mariojv: I generally just edit the wiki and add things to the agenda. I don't think there's an approval process. | 22:27 |
| mariojv | Alright | 22:27 |
| JayF | mariojv: I'd add it to "open discussion" if that makes you feel strange :) | 22:27 |
| JayF | mariojv: if you weren't supposed to, just remember how harsh of a master jroll is | 22:27 |
| JayF | mar:P | 22:27 |
| *** david-ly_ has joined #openstack-ironic | 22:29 | |
| mariojv | /nick mar:P | 22:31 |
| mariojv | "Erroneous nickname" :( | 22:31 |
| *** david-ly_ is now known as david-lyle_ | 22:31 | |
| JayF | mar[tab]: :P | 22:31 |
| NobodyCam | lol | 22:32 |
| *** david-lyle has quit IRC | 22:32 | |
| *** rvasilets___ has joined #openstack-ironic | 22:32 | |
| *** dprince has quit IRC | 22:37 | |
| openstackgerrit | Merged openstack/bifrost: Additional inspection role and adding role to NV job script https://review.openstack.org/232746 | 22:37 |
| *** ionutbalutoiu has quit IRC | 22:40 | |
| *** kkoski has quit IRC | 22:43 | |
| devananda | mariojv: no form. I'll add it | 22:44 |
| mariojv | devananda: already added it to the wiki, but feel free to make any modifications. thanks | 22:44 |
| devananda | mariojv: yea, that doesn't follow our layout template | 22:45 |
| devananda | mariojv: Example topic: (devananda) Let's talk about zebras. Reference: http://en.wikipedia.org/wiki/Zebra | 22:45 |
| devananda | put a line like that under the Discussion heading, pls | 22:45 |
| *** aginwala_ has quit IRC | 22:46 | |
| devananda | actually - this qualifies as a stuck spec | 22:46 |
| devananda | stuck since you need moar input | 22:46 |
| JayF | I think also some decisioneering needs to happen too | 22:46 |
| JayF | i.e. I'm not sure we can talk our way out of some folks wanting bigger notifications vs folks thinking we need smaller | 22:46 |
| JayF | but idk | 22:46 |
| *** penick has quit IRC | 22:48 | |
| *** aginwala has joined #openstack-ironic | 22:49 | |
| openstackgerrit | Arun S A G proposed openstack/ironic-specs: Add spec for deploying OS tarballs https://review.openstack.org/248968 | 22:51 |
| JayF | Ooh, I bet that's interesting | 22:51 |
| devananda | JayF: IPA already can write an MBR, so ^ isn't much of a stretch | 22:54 |
| JayF | Oh absolutely | 22:54 |
| devananda | it comes very close to the area traditionally held by installers, but I think it's still on this side -- there's a binary image (of files, not a filesystem) | 22:55 |
| JayF | I have no problem whatsoever with it in theory | 22:55 |
| *** jaybeale has joined #openstack-ironic | 22:55 | |
| *** [1]cdearborn has quit IRC | 22:57 | |
| *** rajathagasthya has quit IRC | 23:05 | |
| *** dims has quit IRC | 23:10 | |
| *** Sukhdev has quit IRC | 23:12 | |
| *** ionutbalutoiu has joined #openstack-ironic | 23:20 | |
| *** rajathagasthya has joined #openstack-ironic | 23:28 | |
| *** mbound_ has quit IRC | 23:31 | |
| *** openstack has joined #openstack-ironic | 23:38 | |
| *** Sukhdev has joined #openstack-ironic | 23:45 | |
| *** penick has joined #openstack-ironic | 23:45 | |
| *** dims has joined #openstack-ironic | 23:47 | |
| *** rajathagasthya has quit IRC | 23:51 | |
| *** penick has quit IRC | 23:51 | |
| *** penick has joined #openstack-ironic | 23:53 | |
Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!
