| *** zhurong has quit IRC | 00:00 | |
| *** Pavo has joined #openstack-kolla | 00:05 | |
| *** vhosakot has joined #openstack-kolla | 00:15 | |
| *** srwilkers_ has joined #openstack-kolla | 00:16 | |
| *** fguillot has joined #openstack-kolla | 00:18 | |
| *** Jeffrey4l_ has joined #openstack-kolla | 00:20 | |
| *** Jeffrey4l__ has quit IRC | 00:20 | |
| *** vhosakot has quit IRC | 00:25 | |
| sdake_ | Daviey about? | 00:29 |
|---|---|---|
| sdake_ | Jeffrey4l_ ping re osic | 00:30 |
| sdake_ | Jeffrey4l_ i've captured all data for ubuntu | 00:30 |
| sdake_ | now we need to do the same for centos | 00:30 |
| Jeffrey4l_ | sdake_, cool | 00:30 |
| sdake_ | Jeffrey4l_ do you hae any idea how to deploy centos? | 00:30 |
| Jeffrey4l_ | no. :( never touch the deploy OS scripts. | 00:31 |
| sdake_ | it has soething to do with the lxccobbler container | 00:31 |
| Jeffrey4l_ | let me check the doc and the script. | 00:32 |
| *** fguillot has quit IRC | 00:33 | |
| Mech422 | sdake_: regarding https://review.openstack.org/#/c/346214/ - you didn't want to use the $PATH method, and suggest I talk to pburke about it. | 00:45 |
| openstackgerrit | Steven Dake proposed openstack/kolla: Add OSIC Scale Testing Documentation https://review.openstack.org/352101 | 00:45 |
| Mech422 | sdake_: I mentioned it a couple of times without response. | 00:45 |
| Mech422 | sdake_: so we ok with just using $PATH or what ? | 00:46 |
| *** fguillot has joined #openstack-kolla | 00:46 | |
| sdake_ | Mech422 just rebase - and our nv gate will fial | 00:47 |
| sdake_ | and we can resolve that as a separate patch | 00:47 |
| sdake_ | Mech422 if pbourke_ gives the go-ahead we can merge it | 00:47 |
| Mech422 | sdake_: ok - I'll try to get to that later this week | 00:47 |
| sdake_ | even though it breaks the gate | 00:47 |
| Mech422 | sdake_: still doing stuff for $BOSS atm | 00:48 |
| sdake_ | it takes 30 seconds to rebase - like me to do so? :) | 00:48 |
| *** fragatina has joined #openstack-kolla | 00:48 | |
| Mech422 | if you want - I'm actually setting up oVirt atm - after having done cloudstack and archipel | 00:48 |
| Mech422 | boss seems to want to try out everything with 'virtual' in the name | 00:49 |
| sdake_ | cool - hae any feedback from that? | 00:49 |
| *** duonghq has joined #openstack-kolla | 00:49 | |
| sdake_ | Jeffrey4l_ can you chane mtu on two boxes without cratering the environment to 9k? | 00:50 |
| duonghq | sdake_ where did OSIC test go? are you nearly done with this | 00:50 |
| sdake_ | and rerun test? | 00:50 |
| sdake_ | duonghq i am done with ubuntu | 00:50 |
| sdake_ | now ffor centos | 00:50 |
| sdake_ | need someone to deploy centos - he no idea how | 00:50 |
| sdake_ | nd test cases takes about 40 hours tor un | 00:51 |
| sdake_ | we he about 45 hours left | 00:51 |
| Jeffrey4l_ | sdake_, we can try. but change the MTU need change the physical switcher too. | 00:51 |
| Mech422 | sdake_: we had high hopes for cloudstack in our small pop 'clusters'... | 00:51 |
| Jeffrey4l_ | no idea what's the MTU on the switcher sdake_ | 00:51 |
| sdake_ | ok - lets try on two non-essential nodes then? | 00:51 |
| Jeffrey4l_ | ok | 00:51 |
| Mech422 | sdake_: but he seemed to lose patience waiting for me to figure out the cloudstack networking | 00:51 |
| sdake_ | i assume 9k | 00:51 |
| *** v1k0d3n has joined #openstack-kolla | 00:52 | |
| Mech422 | sdake_: archipel is based on xmpp - very thin layer over libvirt, easy to deploy but seems a bit 'chatty' on the network side, and I'm not sure the agents are 'production grade' yet | 00:52 |
| sdake_ | sup v1k0d3n | 00:53 |
| sdake_ | Jeffrey4l_ both ends need to be mtu 9k | 00:54 |
| Jeffrey4l_ | sdake_, it is now. | 00:55 |
| sdake_ | what is the second end? | 00:55 |
| Jeffrey4l_ | 0.23 vs 0.66 | 00:55 |
| sdake_ | ok | 00:55 |
| *** v1k0d3n has quit IRC | 00:57 | |
| Jeffrey4l_ | sdake_, http://paste.openstack.org/show/564397/ | 01:01 |
| openstackgerrit | Ross Krumbeck proposed openstack/kolla: Ansible-ize OpenStack Designate https://review.openstack.org/353261 | 01:02 |
| openstackgerrit | Steven Dake proposed openstack/kolla: Add OSIC Scale Testing Documentation https://review.openstack.org/352101 | 01:05 |
| Daviey | sdake_: here | 01:06 |
| duonghq | sdake: I hope that you can leave a brief comment on implementation direction of bp/ansible-specific-task-become implementation @ https://review.openstack.org/#/c/358539/7 when you have spare time | 01:06 |
| duonghq | sdake_, | 01:07 |
| sdake_ | Daviey - it appears we have run out of forwrad movement | 01:07 |
| sdake_ | Daviey i'm not sure if Jeffrey4l_ can sort out deploying centos or not | 01:07 |
| Daviey | hmm | 01:07 |
| sdake_ | if not, your welome to use the cluster for a full ceph test | 01:07 |
| sdake_ | i think we need pbourke_ and inc0 | 01:08 |
| Daviey | sdake_: What is the issue with centos? | 01:08 |
| sdake_ | they know how to do this | 01:08 |
| *** zhiwei has joined #openstack-kolla | 01:08 | |
| sdake_ | Daviey - not sure how to deploy it to our 130 nodes | 01:08 |
| Daviey | sdake_: Are we using bifrost or still cobbler? | 01:08 |
| sdake_ | britthou_ was working on it | 01:08 |
| sdake_ | cobble | 01:08 |
| Daviey | (I hate cobbler) | 01:09 |
| sdake_ | part of the issue is it is in an lxc thing | 01:09 |
| sdake_ | which i don't know much about ;) | 01:09 |
| *** zhurong has joined #openstack-kolla | 01:10 | |
| *** serverascode has quit IRC | 01:10 | |
| sdake_ | lets wait until Jeffrey4l_ gts back from breakfast and see if he can sort it out | 01:10 |
| sdake_ | if not, we can setup your scenario as #11 | 01:10 |
| sdake_ | since i need cento to proceed | 01:10 |
| *** NachoDuck has quit IRC | 01:11 | |
| Daviey | sdake_: well, it is 2:15am here. I'm reluctant to start something i can't reasonably finish before i need to sleep | 01:15 |
| Pavo | sdake_ you have a link for this bifrost thing you been talking about? | 01:16 |
| *** salv-orlando has joined #openstack-kolla | 01:17 | |
| Daviey | Pavo: bifrost is just ironic, but ansibleised | 01:18 |
| Daviey | and standlone | 01:18 |
| Pavo | hmmm any info on it? | 01:18 |
| Daviey | Pavo: http://docs.openstack.org/developer/bifrost/ | 01:19 |
| Pavo | so this is a way to basically net install OS on bare metal using ironic from Openstack without making apart of resources in openstack? | 01:21 |
| *** salv-orlando has quit IRC | 01:22 | |
| *** NachoDuck has joined #openstack-kolla | 01:24 | |
| *** sdake_ has quit IRC | 01:25 | |
| *** serverascode has joined #openstack-kolla | 01:26 | |
| *** zhubingbing has joined #openstack-kolla | 01:27 | |
| Daviey | Pavo: yeah, ironic standaone | 01:28 |
| mark-casey | Pavo: yes. if I understand correctly this does not necessarily deploy Ironic with the OpenStack deploy. It just means that if you want to deploy OpenStack using Ironic you get a chicken-and-egg problem trying to use an existing Ironic install. Hence Bifrost | 01:28 |
| mark-casey | correction *this does not necessarily _mean you will not_ deploy Ironic | 01:29 |
| Pavo | ok I can see its use | 01:30 |
| *** sdake has joined #openstack-kolla | 01:30 | |
| mark-casey | Pavo: it does make some tradeoffs that require, for example, different methods of security. for example the API has much simpler (if any) auth applied to it, because Keystone isn't there to ask | 01:31 |
| sdake | pavo yes that is correct | 01:31 |
| sdake | Daviey wront file | 01:31 |
| sdake | moemnt reconnecting to vpn | 01:31 |
| Pavo | could see the usefulness to be able to login into alot of bare metal machines using the same keypair | 01:32 |
| Daviey | Pavo: that is the least of the benefit! | 01:33 |
| openstackgerrit | Ryan Hallisey proposed openstack/kolla-kubernetes: Update the kubernetes aio guide https://review.openstack.org/361787 | 01:33 |
| sdake | Daviey is that you on irc | 01:33 |
| Jeffrey4l_ | i am back. | 01:33 |
| Daviey | sdake: yes | 01:33 |
| sdake | rather on tmux | 01:33 |
| sdake | sound slike both of you are back | 01:33 |
| sdake | so here is the deal, i want to see if jeffrey can get a node deployed with centos | 01:33 |
| sdake | if he can, lets proceed down centos path | 01:34 |
| sdake | the scenario tests take about 6 hours to run Daviey | 01:34 |
| Pavo | centos is awesome | 01:34 |
| Pavo | lol | 01:34 |
| Pavo | thats what I use on stable | 01:34 |
| Jeffrey4l_ | let me check the osic deploy doc. | 01:34 |
| sdake | i was trying to texxt inc0 | 01:34 |
| sdake | but he appears mia and I don't hve his cell # | 01:35 |
| Pavo | so... how does kubernetes work with kolla? | 01:35 |
| sdake | pavo - we dont hae a cocrete answer to that yet | 01:35 |
| sdake | Pavo kolla-kubernetes 1.0.0 = occata at best | 01:35 |
| sdake | mosst likley p cycle | 01:35 |
| Pavo | I haven't really understood what kubernetes is anyways lol | 01:35 |
| Pavo | only thing I see from kubernetes is self healing | 01:36 |
| sdake | pavo just stick with ansible version of kolla for now | 01:36 |
| Daviey | sdake: hmm.. so what is happening ? | 01:36 |
| sdake | pavo that is right | 01:36 |
| Daviey | sdake: should i proceed or hold fire? | 01:36 |
| sdake | Daviey - we are redeploying the bare metal with centos if its possible | 01:36 |
| sdake | Daviey Jeffrey4l_ is going to deploy only one node hopefully | 01:36 |
| sdake | if it doesn't work you can do your test case | 01:36 |
| Daviey | sdake: Okay, works for me | 01:36 |
| openstackgerrit | Ross Krumbeck proposed openstack/kolla: Ansible-ize OpenStack Designate https://review.openstack.org/353261 | 01:36 |
| sdake | i'd recommend doing the fio test scenario that jeffrey ha sorted out | 01:36 |
| Pavo | so with self healing if a container crashes or freaks out it will bring up another container to replace it without losing any data, is that correct? | 01:37 |
| Daviey | sdake: can you or Jeffrey4l_ provide an update for my morning, to know what i can do tomorrow? | 01:37 |
| Daviey | sdake: I'm happy to try my scenario on centos or ubuntu | 01:37 |
| sdake | Daviey its your end of day? | 01:37 |
| Daviey | sdake: it is 2:37am, so yeah. | 01:37 |
| sdake | Daviey i think we will kknwo in 1-2 hours if Jeffrey4l_ is able to deploy cenots | 01:37 |
| sdake | Daviey roger - feel the pain mysself | 01:38 |
| sdake | i've been up for what seems like 6 days straight | 01:38 |
| Daviey | sdake: This isn't 'our' cobbler is it? | 01:38 |
| sdake | our as in kolla's? | 01:38 |
| Daviey | sdake: this was provided by osic already, right? | 01:38 |
| *** caowei has joined #openstack-kolla | 01:38 | |
| sdake | yes its the osic cobbler | 01:39 |
| Daviey | good. So hopefully they already have a functional centos config | 01:39 |
| sdake | they dont | 01:39 |
| Daviey | otherwise, 1-2 hours is probably optimistic | 01:39 |
| sdake | britthou_ worked on it or 3-4 hours already | 01:39 |
| Daviey | ok, cool | 01:39 |
| sdake | don't redeploy the deploy node Jeffrey4l_ | 01:40 |
| sdake | plz :) | 01:40 |
| Jeffrey4l_ | sdake, yes. | 01:41 |
| Jeffrey4l_ | will not touch that one | 01:41 |
| sdake | Jeffrey4l_ a preseed is an ubuntu thign i think | 01:44 |
| sdake | i think we need a kickstart or something | 01:44 |
| Jeffrey4l_ | sdake, yes. | 01:44 |
| *** zhubingbing has quit IRC | 01:45 | |
| Daviey | sdake: but for cobbler, it is basically the same hook | 01:45 |
| Jeffrey4l_ | but not familiar with how cobbler profile works. need check the code and some doc | 01:45 |
| Daviey | Jeffrey4l_: don't read cobbler code, it will make you cry | 01:45 |
| sdake | Daviey - do you create a preseed for centos then? | 01:45 |
| Daviey | sdake: well yeah | 01:45 |
| sdake | or a kickstart | 01:45 |
| Jeffrey4l_ | need a ks file | 01:45 |
| Daviey | Have we imported the iso? | 01:46 |
| Daviey | or kernel? | 01:46 |
| *** mark-casey has quit IRC | 01:47 | |
| sdake | iso is imported | 01:48 |
| sdake | iso is in the filesystem atleast | 01:48 |
| sdake | i dont know what importing means | 01:48 |
| Daviey | sdake: cobbler doesn't know about it | 01:49 |
| Daviey | sdake: cobbler only knows about ubuntu here? | 01:49 |
| sdake | right | 01:50 |
| Daviey | (i hate cobbler) | 01:52 |
| sdake | Daviey dns i sbust on these machines i htink | 01:55 |
| sdake | you may have ot setup resolv.conf | 01:55 |
| sdake | to 8.8.8.8 | 01:55 |
| sdake | whisky tango foxtrot | 01:58 |
| *** huikang has joined #openstack-kolla | 01:58 | |
| *** zhurong_ has joined #openstack-kolla | 02:02 | |
| sdake | victory | 02:03 |
| sdake | nothing like a reboot to fix things :) | 02:03 |
| Jeffrey4l_ | i will try my fio test now. | 02:05 |
| *** Pavo has quit IRC | 02:05 | |
| sdake | wonder what that hp raid is about | 02:05 |
| sdake | Jeffrey4l_ not sure if th ecluster is operational atm - but your welcome to try while daviey sorts out a centos deploy | 02:05 |
| *** zhurong has quit IRC | 02:06 | |
| *** Pavo has joined #openstack-kolla | 02:06 | |
| Jeffrey4l_ | tried boot a vm, it works. | 02:06 |
| Daviey | sdake / Jeffrey4l_: hmm, we need a cobbler kickstart file now | 02:06 |
| Daviey | the legacy.ks almost certainly will not work ootb | 02:07 |
| sdake | is there somewehre we can get one | 02:07 |
| sdake | or is it roll our own? | 02:07 |
| Daviey | sdake: Would be good if OSIC already had one | 02:08 |
| sdake | i think thta is us :) | 02:08 |
| Daviey | (i'm surprised there isn't one that is known to work) | 02:08 |
| sdake | Jeffrey4l_ did you see anything in the docs about centos? | 02:08 |
| Jeffrey4l_ | no. | 02:08 |
| Jeffrey4l_ | sdake, Daviey this is the one i used for pxe http://paste.openstack.org/show/564449/ | 02:09 |
| Daviey | https://github.com/os-cloud/osic-ref-impl/blob/master/bare_metal_provisioning.md | 02:09 |
| Jeffrey4l_ | may be help | 02:09 |
| Jeffrey4l_ | just install the os and setup a ssh public key. | 02:09 |
| Daviey | That works | 02:10 |
| Daviey | Jeffrey4l_: do you want to run with that? | 02:10 |
| Daviey | hmm, url looks bad tho? | 02:10 |
| Jeffrey4l_ | Daviey, yep. | 02:11 |
| Jeffrey4l_ | it should work. Just provide what can I help. | 02:11 |
| Jeffrey4l_ | no familiar with cobbler... | 02:11 |
| Jeffrey4l_ | The url should be configuration in cobbler? | 02:11 |
| Jeffrey4l_ | see? there is a $tree variable. | 02:12 |
| Daviey | Jeffrey4l_: well it is just kickstart... that also supports templates | 02:12 |
| Daviey | and snippets | 02:12 |
| Jeffrey4l_ | yep | 02:12 |
| Daviey | Jeffrey4l_: the password variable comes from /etc/cobbler/settings file | 02:14 |
| Daviey | and other $variables | 02:14 |
| sdake | Daviey if we can et centos on there and accessible via password login of some type, we ccan handle the rest of the setup i think | 02:14 |
| sdake | but it woudl be handy if the network got automatically steup ;) | 02:14 |
| Daviey | sdake: well lets just try legacy and see what happens | 02:15 |
| Daviey | we have ilo | 02:15 |
| sdake | doing all nodes? | 02:15 |
| Daviey | lets do 1 first | 02:15 |
| sdake | agree | 02:15 |
| sdake | i dont know cobbler | 02:15 |
| sdake | so dont know how it works - haveo nly used the web ui | 02:15 |
| Jeffrey4l_ | please do not touch the controller node. | 02:15 |
| Jeffrey4l_ | now. | 02:15 |
| sdake | use 23 daviey | 02:15 |
| sdake | Jeffrey4l_ 23 is safe right? | 02:16 |
| Jeffrey4l_ | yep | 02:16 |
| Jeffrey4l_ | it is runing. just he p4p1 nic broken | 02:17 |
| sdake | 24-26 are controller nodes Daviey | 02:19 |
| sdake | 23 should be good to try out | 02:19 |
| *** salv-orlando has joined #openstack-kolla | 02:24 | |
| *** salv-orlando has quit IRC | 02:27 | |
| duonghq | sdake: In my commit https://review.openstack.org/#/c/358539/7 , Jeffrey and inc0 have some comments about my approach. Can you review that? | 02:29 |
| sdake | duonghq occupied atm | 02:30 |
| sdake | know ilo password? | 02:32 |
| *** fguillot has quit IRC | 02:33 | |
| sdake | root:calvincalvin | 02:33 |
| openstackgerrit | Lu lei proposed openstack/kolla: Use six.StringIO https://review.openstack.org/361797 | 02:34 |
| Daviey | sdake: can you look at the ilo console, i don't have java setup on my local machine | 02:35 |
| Daviey | (and .net.. well Linux) | 02:35 |
| sdake | trying - failling | 02:36 |
| sdake | idont have .neteither | 02:36 |
| sdake | java luanch button not working | 02:36 |
| sdake | trying safari | 02:38 |
| sdake | needto update java | 02:38 |
| Daviey | sdake: trying serial console | 02:38 |
| sdake | Daviey how do you know all this stuff :) | 02:39 |
| *** senk has joined #openstack-kolla | 02:39 | |
| Daviey | sdake: :) | 02:40 |
| *** daneyon has joined #openstack-kolla | 02:40 | |
| sdake | the console on ucs gear is annoying to use as well | 02:40 |
| sdake | java crap | 02:40 |
| Daviey | hmm. doesn't look like it net booted | 02:41 |
| sdake | wouoldn't appear so | 02:41 |
| Daviey | sdake: If only we had some oracle people here to help with java | 02:42 |
| sdake | in cobbler dont you hveto tellt he machine to save and reboot orsomhting | 02:42 |
| sdake | i recall on the gui there was some magic button | 02:42 |
| Daviey | sdake: i did, i thought | 02:43 |
| Daviey | try this | 02:43 |
| sdake | esc-@? | 02:44 |
| Daviey | sdake: cobbler sync, copies all the files around | 02:44 |
| sdake | thats it sync | 02:45 |
| Daviey | hang on, i set one time boot from net | 02:45 |
| *** daneyon has quit IRC | 02:45 | |
| Daviey | here we go | 02:45 |
| sdake | nice | 02:45 |
| sdake | now no console ;( | 02:46 |
| Daviey | That you are seeing is because i tail'd -F cobbler log file server side | 02:46 |
| Daviey | but yeah, seems centos doesn't export the serial console by default | 02:47 |
| Daviey | so the bad news is that it is unlikely the preseed is working | 02:47 |
| Daviey | so the kernel command line probably needs help to export the tty over serial console | 02:48 |
| Daviey | or.. get ilo / java working | 02:48 |
| sdake | let me try new java moment | 02:48 |
| *** haplo37__ has quit IRC | 02:49 | |
| Daviey | sdake: I should really go to bed soon, it is silly o'clock | 02:50 |
| *** yuanying has quit IRC | 02:51 | |
| Daviey | sdake: Okay, see the command there to set it to boot from pxe? | 02:51 |
| *** yuanying has joined #openstack-kolla | 02:52 | |
| Daviey | sdake / Jeffrey4l_: Okay, i need to head to bed. If someone else can take over this, it would be good | 02:56 |
| Daviey | let me know how you get on. x | 02:56 |
| Jeffrey4l_ | OK. | 02:56 |
| sdake | how do i get the console again? | 02:57 |
| sdake | and trigger a reboot? | 02:57 |
| Daviey | sdake: i sent it to you in pm | 02:57 |
| sdake | Daviey roger - understand need for sleep :) night | 02:57 |
| sdake | sol activate opens up a serial console? | 02:58 |
| Daviey | chassis bootdev pxe # set boot from local | 02:58 |
| sdake | but it seems to not be ddoing so | 02:58 |
| Daviey | sdake: it should be? | 02:58 |
| sdake | what should be | 02:58 |
| Daviey | it should be opening a serial console | 02:58 |
| * sdake strangles inc0 | 02:58 | |
| Daviey | mc reset warm # power reset | 02:58 |
| sdake | but why is there no bios booting? | 02:59 |
| Daviey | sdake: we saw BIOS POST checks, no? | 02:59 |
| sdake | oh i thought you warm reset it | 02:59 |
| sdake | so we should see the bios again right? | 02:59 |
| Daviey | if it is rebooted yeah | 03:00 |
| sdake | mc reset warm = reboot? | 03:00 |
| Daviey | right | 03:00 |
| sdake | so you rebooted, and no serial console | 03:00 |
| Daviey | hmm | 03:00 |
| Daviey | sdake: power reset, works :) | 03:05 |
| Daviey | right, really going now | 03:06 |
| *** huikang has quit IRC | 03:07 | |
| openstackgerrit | Sridar Kandaswamy proposed openstack/kolla: Customizations for dind https://review.openstack.org/353203 | 03:07 |
| sdake | Jeffrey4l_ how do I get to Daviey's tmux? | 03:09 |
| Jeffrey4l_ | sdake, see my tmux | 03:09 |
| sdake | igot it thanks | 03:10 |
| sdake | you young whipersnappers and your tmux tools | 03:11 |
| sdake | and us old whippersnappers and our ipmi tools | 03:11 |
| sdake | rather old dogs | 03:11 |
| *** rkrum has joined #openstack-kolla | 03:11 | |
| sdake | ;) | 03:11 |
| sdake | well i'm stuck ;) | 03:11 |
| sdake | java console not working | 03:11 |
| *** v1k0d3n has joined #openstack-kolla | 03:13 | |
| *** v1k0d3n has quit IRC | 03:13 | |
| *** dave-mccowan has quit IRC | 03:17 | |
| sdake | whatever we do, lets multiply it by 9.2 | 03:18 |
| *** huikang has joined #openstack-kolla | 03:18 | |
| sdake | pbourke_ when your alive - need centos running on the gear if you can make ethat happen | 03:19 |
| sdake | britthou_ same story | 03:19 |
| sdake | daviey imported centos | 03:19 |
| sdake | I think the kickstart file needs some love but I am not sure | 03:19 |
| sdake | Jeffrey4l_ - do you plan to submit this fio test uptream to rally? | 03:20 |
| Jeffrey4l_ | sdake, yes. | 03:20 |
| Jeffrey4l_ | sdake, another test is `booting multi instances test` | 03:21 |
| sdake | well you hve th gear until centos is on it | 03:21 |
| sdake | whenever that is | 03:21 |
| *** v1k0d3n has joined #openstack-kolla | 03:21 | |
| sdake | i dont nwo how to get centos rolling | 03:21 |
| sdake | daviey imported the imge | 03:22 |
| sdake | and did some magic ipmi commands | 03:22 |
| Jeffrey4l_ | where are we now? does the node boot from pxe? | 03:22 |
| *** huikang has quit IRC | 03:23 | |
| sdake | not really | 03:23 |
| sdake | i dont now what its oding | 03:23 |
| sdake | because i can't get a console | 03:23 |
| Jeffrey4l_ | i tried to get the console by java. bug failed, too. | 03:23 |
| sdake | we can get a console via ipmi | 03:24 |
| sdake | but the kickstart file needs attention to output to ttyS0 | 03:24 |
| Jeffrey4l_ | why? | 03:24 |
| Jeffrey4l_ | is it required? | 03:25 |
| Jeffrey4l_ | cobbler need get the console? | 03:25 |
| sdake | seeing the console woudl let us know if its working | 03:25 |
| sdake | so getting a console is next step | 03:25 |
| Jeffrey4l_ | ok.. | 03:25 |
| sdake | do you know how to get back inside the lxc container? | 03:26 |
| Jeffrey4l_ | yes. | 03:26 |
| sdake | can you join othe rtmux or busy with fio testing? | 03:26 |
| Jeffrey4l_ | sdake, i am in the tmux 0 | 03:27 |
| Jeffrey4l_ | sdake, please try tmux att -t 0 | 03:27 |
| *** huikang has joined #openstack-kolla | 03:31 | |
| sdake | we need to get the kickstart file to have console=ttyS0 in some way | 03:34 |
| Jeffrey4l_ | sdake, pxe config file? | 03:34 |
| sdake | goes in kickstart | 03:34 |
| sdake | i think | 03:35 |
| sdake | maye its pxeconfig | 03:35 |
| *** yuanying has quit IRC | 03:44 | |
| *** aswadr_ has joined #openstack-kolla | 03:45 | |
| sdake | Jeffrey4l_ i want to find the kickstart file under use | 03:46 |
| sdake | so we can have that console=ttys0 in there | 03:46 |
| Jeffrey4l_ | sd i think we need change the pxe menu file directly. the ks control the final OS kernel parameter. | 03:48 |
| Jeffrey4l_ | sdake, ctrl+b ] then you can scroll | 03:48 |
| sdake | i id that | 03:48 |
| sdake | wasn't working | 03:48 |
| openstackgerrit | Hui Kang proposed openstack/kolla: Add missing container image names to build config file https://review.openstack.org/356854 | 03:51 |
| sdake | default_kickstart : /var/lib/cobbler/kickstarts/ubuntu-server.preseed | 03:53 |
| sdake | this is wrong | 03:53 |
| sdake | kernel_options : ksdevice=bootif lang= locale=en_US text interface=p1p1 kssendmac priority=critical | 03:55 |
| sdake | Jeffrey4l_ it hsould be spttting out post stuff now | 03:59 |
| openstackgerrit | Lu lei proposed openstack/kolla: Use six.StringIO https://review.openstack.org/361797 | 04:03 |
| *** unicell has joined #openstack-kolla | 04:14 | |
| *** salv-orlando has joined #openstack-kolla | 04:18 | |
| *** senk has quit IRC | 04:19 | |
| *** salv-orlando has quit IRC | 04:23 | |
| duonghq | sdake: for scroll in tmux: C-b [ | 04:25 |
| *** coolsvap_ has joined #openstack-kolla | 04:28 | |
| *** unicell has quit IRC | 04:28 | |
| *** coolsvap_ is now known as coolsvap_\ | 04:28 | |
| *** coolsvap_\ is now known as coolsvap | 04:28 | |
| *** unicell has joined #openstack-kolla | 04:28 | |
| *** mark-casey has joined #openstack-kolla | 04:42 | |
| openstackgerrit | Duong Ha-Quang proposed openstack/kolla: Fix Kibana image build error https://review.openstack.org/361833 | 04:44 |
| openstackgerrit | Merged openstack/kolla: Customizations for Nova https://review.openstack.org/351676 | 04:44 |
| *** fragatin_ has joined #openstack-kolla | 04:45 | |
| openstackgerrit | Swapnil Kulkarni (coolsvap) proposed openstack/kolla: Update Dockerfiles for footer related changes https://review.openstack.org/357746 | 04:46 |
| *** fragati__ has joined #openstack-kolla | 04:48 | |
| *** fragatina has quit IRC | 04:48 | |
| *** fragatin_ has quit IRC | 04:50 | |
| *** fragati__ has quit IRC | 04:53 | |
| *** senk has joined #openstack-kolla | 04:57 | |
| openstackgerrit | Sridar Kandaswamy proposed openstack/kolla: Customizations for dind https://review.openstack.org/353203 | 04:58 |
| sdake | coolsvap do you know anything about cobbler setup in osic? | 05:03 |
| coolsvap | sdake: no | 05:03 |
| *** yuanying has joined #openstack-kolla | 05:03 | |
| coolsvap | but I know there are issues with rolling out centos using cobbler in osic | 05:04 |
| coolsvap | do not know the details | 05:04 |
| sdake | you found these issues from whom? | 05:04 |
| coolsvap | there was a discussion on channel at start between inc0, britt and paul | 05:05 |
| *** v1k0d3n has quit IRC | 05:05 | |
| coolsvap | when we first rolled out ubuntu on all nodes | 05:05 |
| sdake | ya - centos is not preconfigured | 05:07 |
| *** sdake_ has joined #openstack-kolla | 05:10 | |
| *** sdake_ is now known as dake | 05:13 | |
| *** sdake has quit IRC | 05:13 | |
| *** huikang has quit IRC | 05:13 | |
| *** dake is now known as sdake | 05:13 | |
| *** ChanServ sets mode: +o sdake | 05:13 | |
| *** sdake changes topic to "OSIC cluster testing etherpad https://goo.gl/vAMGB6 - Kolla IRC meetngs on Wednesdays @ 16:00 UTC - see agenda @ https://goo.gl/OXB0DL - IRC channel is *LOGGED* @ http://goo.gl/3mzZ7b (old logs from #kolla http://goo.gl/VKpPzA)" | 05:15 | |
| *** sdake changes topic to "ALL HANDS SETUP CENTOS ON OSIC - OSIC cluster testing etherpad https://goo.gl/vAMGB6 - Kolla IRC meetngs on Wednesdays @ 16:00 UTC - see agenda @ https://goo.gl/OXB0DL - IRC channel is *LOGGED* @ http://goo.gl/3mzZ7b (old logs from #kolla http://goo.gl/VKpPzA)" | 05:15 | |
| sdake | bedtime | 05:15 |
| * sdake ptfos | 05:16 | |
| duonghq | bye sdake | 05:16 |
| *** shardy has joined #openstack-kolla | 05:17 | |
| *** daneyon has joined #openstack-kolla | 05:23 | |
| *** sdake has quit IRC | 05:26 | |
| *** daneyon has quit IRC | 05:27 | |
| openstackgerrit | Eduardo Gonzalez proposed openstack/kolla: Change source with dot at extend_start files https://review.openstack.org/359926 | 05:31 |
| *** salv-orlando has joined #openstack-kolla | 05:36 | |
| openstackgerrit | Eduardo Gonzalez proposed openstack/kolla: Remove RUN macro install from Kibana Dockerfile https://review.openstack.org/361846 | 05:43 |
| *** ssurana has joined #openstack-kolla | 05:51 | |
| *** duonghq_ has joined #openstack-kolla | 06:00 | |
| *** duonghq has quit IRC | 06:03 | |
| *** salv-orlando has quit IRC | 06:05 | |
| *** ssurana has quit IRC | 06:06 | |
| *** salv-orlando has joined #openstack-kolla | 06:07 | |
| openstackgerrit | OpenStack Proposal Bot proposed openstack/kolla: Updated from global requirements https://review.openstack.org/361865 | 06:11 |
| *** matrohon has quit IRC | 06:17 | |
| *** egonzalez90 has joined #openstack-kolla | 06:18 | |
| bjolo | morning | 06:22 |
| *** fragatina has joined #openstack-kolla | 06:25 | |
| *** fragatina has quit IRC | 06:26 | |
| *** fragatina has joined #openstack-kolla | 06:26 | |
| *** yuanying has quit IRC | 06:29 | |
| *** mdnadeem_ has quit IRC | 06:37 | |
| *** mdnadeem_ has joined #openstack-kolla | 06:37 | |
| *** Mr_Broken has joined #openstack-kolla | 06:39 | |
| duonghq_ | jenkins gates failed again, no idea | 06:51 |
| *** duonghq_ is now known as duonghq | 06:51 | |
| openstackgerrit | Tin Lam proposed openstack/kolla: Chage with_ loop variables syntax to ansible 2 https://review.openstack.org/361886 | 06:53 |
| *** aswadr_ has quit IRC | 07:01 | |
| openstackgerrit | Matthew Taylor proposed openstack/kolla: Fix nova_ssh container shell to allow instance resizes. https://review.openstack.org/361891 | 07:03 |
| *** mdnadeem_ has quit IRC | 07:12 | |
| *** mdnadeem_ has joined #openstack-kolla | 07:12 | |
| egonzalez90 | Can someone else review this PS? https://review.openstack.org/#/c/361833/ | 07:13 |
| *** zhubingbing has joined #openstack-kolla | 07:14 | |
| zhubingbing | - -hello | 07:14 |
| openstackgerrit | zhubingbing proposed openstack/kolla: repair gnocchi dockerfile https://review.openstack.org/348388 | 07:14 |
| zhubingbing | ｐｉｎｇEduardo Gonzalez | 07:16 |
| egonzalez90 | hi zhubingbing | 07:16 |
| zhubingbing | ping Eduardo Gonzalez | 07:16 |
| zhubingbing | hello | 07:16 |
| zhubingbing | https://review.openstack.org/348388 | 07:16 |
| zhubingbing | i have fixed it , | 07:17 |
| zhubingbing | can you help review it ? | 07:17 |
| openstackgerrit | zhubingbing proposed openstack/kolla: repair aodh image dockerfile https://review.openstack.org/350715 | 07:24 |
| *** mewald has joined #openstack-kolla | 07:26 | |
| *** shardy has quit IRC | 07:26 | |
| *** Mr_Broken has quit IRC | 07:29 | |
| *** matrohon has joined #openstack-kolla | 07:34 | |
| *** rkrum has quit IRC | 07:35 | |
| zhubingbing | ping Hui Kang | 07:37 |
| openstackgerrit | Jeffrey Zhang proposed openstack/kolla: Fix the wrong install_packages usage in kibana Dockerfile https://review.openstack.org/361909 | 07:38 |
| openstackgerrit | Jeffrey Zhang proposed openstack/kolla: Fix removing nova_libvirt container issue https://review.openstack.org/361728 | 07:39 |
| *** duonghq has quit IRC | 07:39 | |
| *** berendt has joined #openstack-kolla | 07:45 | |
| berendt | Morning everybody. | 07:46 |
| zhubingbing | morning | 07:47 |
| *** yuanying has joined #openstack-kolla | 07:51 | |
| openstackgerrit | Merged openstack/kolla: Fix Kibana image build error https://review.openstack.org/361833 | 07:52 |
| *** frzn__ has joined #openstack-kolla | 08:05 | |
| *** frzn__ has quit IRC | 08:06 | |
| openstackgerrit | Mathias proposed openstack/kolla: Added influxdb role https://review.openstack.org/346449 | 08:12 |
| mewald | Hey guys, I'll have time to work on the whole telegraf + influxdb + grafana thing this week before I go on vacation. Would be awesome to have fast review cycles to get it merged this week :) Sorry for not being on it recently … was busy with customers and stuff ^^ | 08:14 |
| *** caowei has quit IRC | 08:17 | |
| *** coolsvap is now known as coolsvap_ | 08:21 | |
| *** duonghq has joined #openstack-kolla | 08:23 | |
| *** zhubingbing has quit IRC | 08:24 | |
| *** mark-casey has quit IRC | 08:26 | |
| *** zhubingbing_ has joined #openstack-kolla | 08:26 | |
| berendt | zhubingbing_ can you please check https://review.openstack.org/#/c/352120/2 and remove your -1 | 08:27 |
| *** zhubingbing_ has quit IRC | 08:28 | |
| *** zhubingbing_ has joined #openstack-kolla | 08:29 | |
| zhubingbing_ | sure i will review it | 08:33 |
| openstackgerrit | Jeffrey Zhang proposed openstack/kolla: Fix removing nova_libvirt container issue https://review.openstack.org/361728 | 08:37 |
| *** rmart04 has joined #openstack-kolla | 08:39 | |
| pbourke_ | morning | 08:43 |
| egonzalez90 | morning pbourke_ | 08:46 |
| pbourke_ | it seems like sdake would like osic reinstalled now with centos | 08:46 |
| pbourke_ | im not sure how difficult that will be but will give it a go soon if anyone wants to help | 08:47 |
| openstackgerrit | Mathias proposed openstack/kolla: Added influxdb role https://review.openstack.org/346449 | 08:48 |
| *** godleon has joined #openstack-kolla | 08:49 | |
| *** salv-orlando has quit IRC | 08:49 | |
| *** godleon has quit IRC | 08:49 | |
| openstackgerrit | Mathias proposed openstack/kolla: Added influxdb role https://review.openstack.org/346449 | 08:53 |
| *** gfidente has joined #openstack-kolla | 08:53 | |
| *** gfidente has quit IRC | 08:54 | |
| *** gfidente has joined #openstack-kolla | 08:54 | |
| openstackgerrit | Lu lei proposed openstack/kolla: Fix using filter() to meet python2,3 https://review.openstack.org/361950 | 08:56 |
| openstackgerrit | Merged openstack/kolla-kubernetes: Cleanup tox.ini constraints https://review.openstack.org/360984 | 08:58 |
| *** duonghq has left #openstack-kolla | 08:58 | |
| *** daneyon has joined #openstack-kolla | 08:59 | |
| *** pbourke_ has quit IRC | 09:00 | |
| *** rmart04 has quit IRC | 09:01 | |
| *** daneyon has quit IRC | 09:04 | |
| openstackgerrit | Prithiv proposed openstack/kolla: Dockerfiles for collectd Container https://review.openstack.org/358632 | 09:09 |
| *** wznoinsk_ is now known as wznoinsk | 09:09 | |
| *** dwalsh has joined #openstack-kolla | 09:12 | |
| *** pbourke has joined #openstack-kolla | 09:20 | |
| *** salv-orlando has joined #openstack-kolla | 09:20 | |
| * Daviey checks in | 09:23 | |
| *** rmart04 has joined #openstack-kolla | 09:27 | |
| *** msimonin has joined #openstack-kolla | 09:28 | |
| Daviey | pbourke: Hey. So last night i set up cobbler to support centos.. but i don't think the ks was suitable | 09:30 |
| pbourke | Daviey: ok | 09:30 |
| Daviey | pbourke: not sure if sdake did anything after i went to sleep | 09:30 |
| pbourke | Daviey: i dont think so | 09:31 |
| pbourke | Daviey: where is the kickstart | 09:31 |
| Daviey | In cobbler | 09:31 |
| Daviey | pbourke: cobbler system getgs --name=SERVERNAME | 09:32 |
| Daviey | getks* | 09:32 |
| pbourke | fair bit in there | 09:34 |
| *** flaper87 has quit IRC | 09:35 | |
| Daviey | pbourke: make sure you are looking at a node using profile centos., otherwise you are looking at a ubuntu preseed | 09:36 |
| *** rmart04 has quit IRC | 09:38 | |
| *** rkrum has joined #openstack-kolla | 09:42 | |
| *** rkrum has quit IRC | 09:47 | |
| *** caowei has joined #openstack-kolla | 09:48 | |
| *** dave-mccowan has joined #openstack-kolla | 09:50 | |
| *** Satya has joined #openstack-kolla | 09:51 | |
| caowei | hi coolsvap | 09:51 |
| Satya | Hi All | 09:51 |
| Satya | i was trying to do a sudo on nova_compute | 09:52 |
| caowei | https://review.openstack.org/#/c/355408/ | 09:52 |
| Satya | not sure if there is a default password | 09:52 |
| Satya | how can i get the password of the nova user in the nova_compute container? | 09:52 |
| *** mnasiadka has joined #openstack-kolla | 09:52 | |
| *** flaper87 has joined #openstack-kolla | 09:53 | |
| *** flaper87 has quit IRC | 09:54 | |
| *** flaper87 has joined #openstack-kolla | 09:54 | |
| *** zigo_ is now known as zigo | 09:56 | |
| pbourke | Daviey: have you tried kickstarting a machine with this profile? | 09:57 |
| openstackgerrit | Merged openstack/kolla: Updated from global requirements https://review.openstack.org/361865 | 10:00 |
| *** zhubingbing_ has quit IRC | 10:03 | |
| *** zhurong_ has quit IRC | 10:04 | |
| *** coolsvap_ is now known as coolsvap | 10:06 | |
| *** Satya has quit IRC | 10:09 | |
| *** egonzalez90 has quit IRC | 10:12 | |
| openstackgerrit | Merged openstack/kolla: Change source with dot at extend_start files https://review.openstack.org/359926 | 10:15 |
| *** zhubingbing_ has joined #openstack-kolla | 10:15 | |
| coolsvap | pbourke: ping | 10:18 |
| pbourke | coolsvap: hieulq | 10:18 |
| pbourke | coolsvap: *hi | 10:18 |
| pbourke | :) | 10:18 |
| pbourke | autocompleted by accident there | 10:18 |
| coolsvap | pbourke: regarding the footer review | 10:18 |
| coolsvap | though deprecated we need to keep the {{ include footer }} blocks for backward compat | 10:19 |
| coolsvap | and inc0 mentioned there should be no unnamed footer blocks | 10:19 |
| pbourke | coolsvap: there needs to be unnamed ones in non base images | 10:21 |
| pbourke | coolsvap: he's incorrect - how else do I unset the proxy for example | 10:21 |
| *** zhiwei has quit IRC | 10:22 | |
| coolsvap | so it is needed for a change that can be applied across all images like e.g. unset proxy | 10:23 |
| Daviey | pbourke: honestly, the proxy thing is a distraction.. why bother putting it into dockerfile templating, when you can just set and unset proxy settings in the shell? | 10:23 |
| coolsvap | so we have two different footer blocks | 10:23 |
| coolsvap | both unnamed | 10:24 |
| pbourke | Daviey: there's other use cases, it's about setting something for the duration of the build | 10:24 |
| pbourke | coolsvap: yes | 10:24 |
| pbourke | Daviey: but not having it persist in the final images | 10:24 |
| Daviey | pbourke: right... but the proxy example everyone bounces around is annoying. | 10:24 |
| Daviey | pbourke: If we have better examples, we can solve it better :) | 10:25 |
| pbourke | potentially it could be removed in favour of build-args | 10:26 |
| pbourke | there probably should have been a spec for the customisations | 10:26 |
| pbourke | pretty messy as is | 10:26 |
| berendt | is there a documentation about the proper use of the footers in the dockerfiles? | 10:27 |
| berendt | if not maybe we should first write the docs..? | 10:27 |
| pbourke | berendt: I have a patchset up | 10:27 |
| pbourke | berendt: I also just sent a link to the ML | 10:27 |
| berendt | let's review and merge the docs first and continue with the pending reviews afterwards | 10:27 |
| pbourke | +1 | 10:28 |
| pbourke | Daviey: any idea how close the current centos ks is to working? | 10:28 |
| pbourke | Daviey: has it being modified at all? | 10:28 |
| berendt | pbourke can you link the review here | 10:29 |
| pbourke | https://review.openstack.org/#/c/361253/2/doc/CONTRIBUTING.rst | 10:29 |
| openstackgerrit | Paul Bourke proposed openstack/kolla: Add guidelines on adding a new service https://review.openstack.org/361253 | 10:30 |
| berendt | pbourke I noted some syntax issues | 10:33 |
| openstackgerrit | Paul Bourke proposed openstack/kolla: Add guidelines on adding a new service https://review.openstack.org/361253 | 10:35 |
| *** caowei has quit IRC | 10:36 | |
| openstackgerrit | Christian Berendt proposed openstack/kolla: Fix using filter() to meet python2,3 https://review.openstack.org/361950 | 10:39 |
| Daviey | pbourke: well.. the installer starts and the kickstart is pulled in | 10:45 |
| openstackgerrit | Lu lei proposed openstack/kolla: Fix using filter() to meet python2,3 https://review.openstack.org/361950 | 10:45 |
| Daviey | i couldn't easiy use the iLo client last night.. (java bad version and .net, well linux desktop) | 10:45 |
| Daviey | so i was using serial console, and centos wasn't setup to export the progress | 10:45 |
| Daviey | so it just needs someone to look at the iLo vnc like interface and see where it is blocking | 10:46 |
| *** Mech422 has quit IRC | 10:47 | |
| pbourke | i'll try take a look | 10:53 |
| pbourke | Daviey: which node? | 10:53 |
| Daviey | hmm | 11:03 |
| Daviey | pbourke: I was using 729583-comp-s3700-023.cloud7.osic.rackspace.com | 11:05 |
| Daviey | as a test node | 11:05 |
| *** zhubingbing_ has quit IRC | 11:16 | |
| openstackgerrit | dupengfei proposed openstack/kolla: * Fix lost parameter --remote in generated start_ovsdb_server.sh https://review.openstack.org/362027 | 11:20 |
| *** v1k0d3n has joined #openstack-kolla | 11:28 | |
| *** zhubingbing_ has joined #openstack-kolla | 11:29 | |
| *** v1k0d3n has quit IRC | 11:33 | |
| *** egonzalez90 has joined #openstack-kolla | 11:40 | |
| *** daneyon has joined #openstack-kolla | 11:41 | |
| *** daneyon has quit IRC | 11:46 | |
| *** shardy has joined #openstack-kolla | 11:50 | |
| *** shardy_ has joined #openstack-kolla | 11:52 | |
| *** shardy_ has quit IRC | 11:52 | |
| *** zhubingbing_ has quit IRC | 11:54 | |
| mewald | please review https://review.openstack.org/#/c/346449/19 :) | 11:56 |
| openstackgerrit | Lu lei proposed openstack/kolla: Fix oslo.i18n in kolla project https://review.openstack.org/355342 | 12:00 |
| *** dwalsh has quit IRC | 12:01 | |
| *** dave-mcc_ has joined #openstack-kolla | 12:07 | |
| *** salv-orl_ has joined #openstack-kolla | 12:08 | |
| *** aswadr_ has joined #openstack-kolla | 12:08 | |
| *** dave-mccowan has quit IRC | 12:09 | |
| *** salv-orlando has quit IRC | 12:10 | |
| *** rmart04 has joined #openstack-kolla | 12:14 | |
| mewald | also need reviews for https://review.openstack.org/#/c/346448/ | 12:15 |
| *** salv-orl_ has quit IRC | 12:16 | |
| *** zhurong has joined #openstack-kolla | 12:18 | |
| *** rhallisey has joined #openstack-kolla | 12:19 | |
| *** rmart04 has quit IRC | 12:26 | |
| *** salv-orlando has joined #openstack-kolla | 12:29 | |
| openstackgerrit | Mathias proposed openstack/kolla: Added influxdb role https://review.openstack.org/346449 | 12:32 |
| openstackgerrit | Mathias proposed openstack/kolla: Added telegraf role https://review.openstack.org/346448 | 12:33 |
| *** v1k0d3n has joined #openstack-kolla | 12:42 | |
| openstackgerrit | Lu lei proposed openstack/kolla: Fix oslo.i18n in kolla project https://review.openstack.org/355342 | 12:44 |
| openstackgerrit | Sridar Kandaswamy proposed openstack/kolla: Customizations for dind https://review.openstack.org/353203 | 13:01 |
| *** sdake has joined #openstack-kolla | 13:03 | |
| sdake | pbourke any luck with centos on osic? | 13:05 |
| pbourke | sdake: looking at it breifly this morning with Daviey | 13:05 |
| pbourke | sdake: need to figure out why the kickstart isn't working, I haven't had a chance to check yet | 13:05 |
| *** strigazi_AFK is now known as strigazi | 13:05 | |
| sdake | is britthou_ in? | 13:07 |
| *** lrensing_ has joined #openstack-kolla | 13:07 | |
| sdake | if we ont have centos soon wewont have any centos at all | 13:07 |
| Daviey | sdake: I'm distracted today.. so don't rely on me | 13:09 |
| Daviey | I think TBH, the centos test isn't going to happen | 13:10 |
| sdake | Daviey well your right it isn't going to do itself.. | 13:10 |
| sdake | we have roughly 30 hours left of osic time - i had really expected centos to be installed when I woke up this morning | 13:11 |
| sdake | i wouold do it myself if I knew hwo, but I don't know how t odo so | 13:12 |
| Daviey | Did anyone get onto the iLo display? | 13:12 |
| sdake | Jeffrey4l_ was unable to doso | 13:13 |
| Daviey | unable?! | 13:13 |
| sdake | i triedturning on ttyS0 from kernel command line | 13:13 |
| sdake | Daviey the java didn't work for him | 13:13 |
| Daviey | *sigh* | 13:13 |
| sdake | that ilo java thing is a big gPILE OF SHIT | 13:13 |
| sdake | console=ttyS0 had no effect | 13:14 |
| sdake | did 23 not build with centos last night? | 13:14 |
| *** egonzalez90 has quit IRC | 13:16 | |
| openstackgerrit | Serguei Bezverkhi proposed openstack/kolla-kubernetes: Getting NOVA pieces together to launch VM https://review.openstack.org/358911 | 13:17 |
| *** shardy has quit IRC | 13:17 | |
| *** zhiwei has joined #openstack-kolla | 13:18 | |
| sdake | serial console=tty0 console=ttyS0,115200n8 | 13:18 |
| *** zhiwei has quit IRC | 13:22 | |
| Daviey | sdake: that would work on debian / ubuntu, no idea about centos | 13:23 |
| *** sdake_ has joined #openstack-kolla | 13:23 | |
| sdake_ | Jeffrey4l_ are you running a test atm? | 13:24 |
| Jeffrey4l_ | sdake_, yep | 13:24 |
| sdake_ | Jeffrey4l_ when wil it complete | 13:25 |
| Jeffrey4l_ | less than 30 mins | 13:25 |
| sdake_ | will there be rally data? | 13:25 |
| *** jtriley has joined #openstack-kolla | 13:25 | |
| *** sdake has quit IRC | 13:25 | |
| Jeffrey4l_ | sorry? sdake_ | 13:26 |
| sdake_ | Jeffrey4l_ will there be rally data after the test | 13:26 |
| Jeffrey4l_ | yes. | 13:26 |
| Jeffrey4l_ | the io test reasult | 13:27 |
| sdake_ | ok i guess that can be scenario #11 | 13:27 |
| *** huikang has joined #openstack-kolla | 13:27 | |
| irtermite | how goes, folks? | 13:28 |
| Jeffrey4l_ | ok. the result will be a single html file and a single json file. | 13:28 |
| sdake_ | irtermite ubuntu testing is aboutwrapped up - however, there is no cookbook for centos for osic | 13:28 |
| sdake_ | irtermite so we are making one it appears | 13:29 |
| sdake_ | or not making one depending on your interpretation | 13:29 |
| irtermite | cookbook for osic? you mean for the osic environment? | 13:30 |
| irtermite | I see... | 13:30 |
| sdake_ | irtermite i mean to launch centos on the nodes | 13:30 |
| irtermite | ...just a reminder, tomorrow is the last day | 13:31 |
| irtermite | got ya | 13:31 |
| sdake_ | i speent 4 hours jeriking aroudn with it last night before i hit hte wall | 13:31 |
| sdake_ | yes i know | 13:31 |
| sdake_ | re last day - tuesday morning, evening what time exacty? | 13:31 |
| openstackgerrit | Paul Bourke proposed openstack/kolla: Add a check before wiping Ceph disks in bootstrap https://review.openstack.org/337594 | 13:32 |
| irtermite | typically, it would be morning of due date (ie: midnight tonight) but, I may be able to stretch it into the business hours of the day | 13:32 |
| Daviey | irtermite: Adding a few days would be great :) | 13:32 |
| irtermite | Daviey: sorry, but that is not an option. You all already had an extra week from the start, and we have to get it rebuilt for the next group in line. | 13:33 |
| irtermite | =( | 13:33 |
| Daviey | irtermite: What do other users tend to use to deploy the OS? | 13:33 |
| irtermite | sdake_: tomorrow afternoon is fine by me | 13:33 |
| irtermite | Daviey: it all depends on the user and their need, why do you ask? do you need assistance with something? | 13:34 |
| *** lyncos has joined #openstack-kolla | 13:34 | |
| sdake_ | yes getting centos to deploy would be fantastic | 13:34 |
| sdake_ | osic-prep doesn't include such support | 13:34 |
| Daviey | irtermite: Well, so far all of the tests have been Ubuntu based | 13:34 |
| Daviey | irtermite: we wanted to smash out a few centos tests | 13:34 |
| openstackgerrit | Paul Bourke proposed openstack/kolla: Add a check before wiping Ceph disks in bootstrap https://review.openstack.org/337594 | 13:35 |
| irtermite | personally, we tend to build a cobbler system on one of the nodes and use that to push all the systems | 13:35 |
| Daviey | irtermite: right | 13:35 |
| sdake_ | irtermite the cobbler system you build (osic-prep) has no implementation for centos | 13:35 |
| Daviey | irtermite: Do you have a centos kickstart template that is known to work? | 13:35 |
| irtermite | I built | 13:35 |
| irtermite | ? | 13:35 |
| irtermite | we did not provide you with any such thing. | 13:35 |
| sdake_ | you/osic | 13:35 |
| sdake_ | someone provided us with osic-prep ;) | 13:36 |
| sdake_ | its an lxc container with cobbler buit in | 13:36 |
| irtermite | sorry, we provide you with a blank slate. if there is anything installed on the environment that was one of the kolla members working on this project with you | 13:36 |
| sdake_ | pretty ure osic-prep comes from the directions | 13:36 |
| irtermite | what directions? | 13:37 |
| sdake_ | Daviey where were those osic-prep instrutions? | 13:37 |
| Daviey | github | 13:37 |
| sdake_ | daviey something more specific as in a link? | 13:37 |
| Daviey | sdake_: https://github.com/os-cloud/osic-ref-impl/blob/master/bare_metal_provisioning.md | 13:37 |
| irtermite | ah that | 13:37 |
| irtermite | yes, that is what we use for cleaning up and re-building the environemnt | 13:37 |
| sdake_ | it doesn't build centos environments.... | 13:38 |
| irtermite | it isn't necessarily something we 'provide' for the users; though we do share it from time to time. Michal probably shared that with you all | 13:38 |
| sdake_ | Daviey Jeffrey4l_ how do i connect to the osic prep container | 13:39 |
| Jeffrey4l_ | sdake_, lxc-ls -f : show all lxc | 13:39 |
| irtermite | correct. that is a rackspace tool that we use to clean up the environment. it wasn't intended for any kind of production or other os use. but, it would be cool to see it start to grow for others to use more regularly | 13:39 |
| Daviey | sdake_: tmux 0 ? | 13:39 |
| Jeffrey4l_ | sdake_, lxc-attach -n osic-prep : attach the lxc | 13:40 |
| irtermite | I actually shared it with another team that was using ubuntu. and their feedback was that it was helpful | 13:40 |
| sdake_ | irtermite works well enough on ubuntu | 13:40 |
| sdake_ | no centos implementation in it at all | 13:40 |
| irtermite | if you all do plan on adding centos support to it, I would certainly appreciate a pull request to help future users of the clusters! | 13:40 |
| sdake_ | no doubt | 13:40 |
| irtermite | this is great feedback too | 13:41 |
| *** inc0 has joined #openstack-kolla | 13:41 | |
| irtermite | I can certainly see a need for a repo of tools for others that snag access | 13:41 |
| *** banix has joined #openstack-kolla | 13:42 | |
| inc0 | good morning | 13:42 |
| irtermite | look who decided to join us | 13:42 |
| irtermite | ;) | 13:42 |
| inc0 | uh... | 13:42 |
| inc0 | ok, hit me, wassup? | 13:42 |
| sdake_ | inc0 hope you had a nice peaceful weekend | 13:42 |
| sdake_ | inc0 need centos on the nodes | 13:43 |
| irtermite | notta. just giving you crap, inc0 | 13:43 |
| Daviey | inc0: Hope you are well rested :) | 13:43 |
| sdake_ | ya i'm not well rested and in a really angry mood atm | 13:43 |
| Daviey | sdake_: Hmm... we might run into problems with centos | 13:43 |
| Daviey | (and might be the issues we saw) | 13:43 |
| sdake_ | which type of problems? | 13:43 |
| Daviey | The ubuntu image is mofified to include i40e driver | 13:44 |
| Daviey | I bet centos doesn't have this | 13:44 |
| sdake_ | the first step is getting a console log | 13:44 |
| irtermite | sdake_: I believe you just need to create a new preseed/kickstart file and upload a centos image, then you should be good to go | 13:44 |
| sdake_ | i40e is not in kernel? | 13:44 |
| inc0 | Daviey, nah, i40n comes with new kernel | 13:44 |
| inc0 | i40e | 13:44 |
| sdake_ | irtermite eaier said then done | 13:44 |
| Daviey | ok, cool | 13:44 |
| Daviey | Someone needs to use windows to connect to the iLo | 13:44 |
| Daviey | I have a meeting now, but if not - i will spin up a VM with windows on after | 13:45 |
| sdake_ | irtermite - yell at the hp dudes to fix their java implementation of console btw ;) | 13:45 |
| sdake_ | irtermite what htey have donesn't work on mac or linux in any browser | 13:45 |
| irtermite | sdake_: and create centos drivers? yea... after I finish yelling at them for their raid firmware BS | 13:46 |
| irtermite | sdake_: works perfectly fine on safari | 13:46 |
| sdake_ | nah this is an LO thing | 13:46 |
| inc0 | yeah safari is ok | 13:46 |
| sdake_ | ilo thing | 13:46 |
| sdake_ | doesn't work for me | 13:46 |
| *** ayoung has joined #openstack-kolla | 13:47 | |
| sdake_ | not the f5 firewlal, but the ilo console | 13:47 |
| irtermite | yea, I know it's an ilo thing... was just saying I've been yelling at them about something else for a few months now | 13:47 |
| sdake_ | i worked on raid in my early career | 13:47 |
| sdake_ | my only recommendation would be use JBOD with software raid these days ;-) | 13:47 |
| irtermite | sdake_: ilo on safari works for me. what is the trouble you are seeing | 13:48 |
| sdake_ | intel integrated raid - what a charlie foxtrot that program was | 13:48 |
| irtermite | inc0: give this a try https://www.centosblog.com/centos-7-minimal-kickstart-file/ | 13:49 |
| inc0 | uhh...that's gonna be fun | 13:49 |
| *** sdake_ has quit IRC | 13:50 | |
| *** sdake has joined #openstack-kolla | 13:51 | |
| inc0 | damn, mass bare metal provisioning is fubar, we need something bifrosty | 13:51 |
| openstackgerrit | Merged openstack/kolla: Add missing container image names to build config file https://review.openstack.org/356854 | 13:54 |
| sdake | inc0 there are two tmux sessions | 13:55 |
| sdake | inc0 connect to mine pls | 13:55 |
| Jeffrey4l_ | sdake i opened the console http://storage7.static.itmages.com/i/16/0829/h_1472479081_3201804_2be9cb4902.png | 13:57 |
| sdake | is that on node 23 Jeffrey4l_ ? | 13:58 |
| Jeffrey4l_ | sdake, yep | 13:58 |
| sdake | did you just rboot it? | 13:58 |
| sdake | we need the os console | 13:58 |
| Jeffrey4l_ | you can see the ip address. | 13:58 |
| Jeffrey4l_ | sdake, i do nothing. just connect it. | 13:59 |
| sdake | whichmeans we need this in the boot command lien: serial console=tty0 console=ttyS0,115200n8 | 13:59 |
| inc0 | sdake, caffeine is yet to kick in | 13:59 |
| sdake | inc0 http://i2.kym-cdn.com/entries/icons/original/000/003/730/cocaine-bear.jpg | 14:00 |
| inc0 | sdake, how did tests go btw? | 14:02 |
| *** fguillot has joined #openstack-kolla | 14:02 | |
| *** jtriley_ has joined #openstack-kolla | 14:03 | |
| *** senk has quit IRC | 14:04 | |
| *** b_bezak has joined #openstack-kolla | 14:05 | |
| sdake | inc0 testing is done on ubuntu | 14:05 |
| sdake | inc0 hwich is why we needed centos yesterday not today | 14:05 |
| sdake | note the kickstart file referenced in the osic config is not correct | 14:06 |
| sdake | and cobbler needs the kernel command line appended with 729517-comp-s3500-023.cloud7.osic.rackspace.com | 14:06 |
| *** jtriley has quit IRC | 14:06 | |
| *** mkoderer has quit IRC | 14:06 | |
| sdake | rather that serial line I pastd above | 14:06 |
| *** lmiccini has quit IRC | 14:07 | |
| sdake | thattook all of 3 minutes | 14:07 |
| sdake | now for serial console | 14:07 |
| *** caowei has joined #openstack-kolla | 14:08 | |
| *** mkoderer has joined #openstack-kolla | 14:08 | |
| *** lmiccini has joined #openstack-kolla | 14:14 | |
| Daviey | back | 14:15 |
| *** zhubingbing has joined #openstack-kolla | 14:15 | |
| zhubingbing | ping Mathias | 14:16 |
| sbezverk_ | inc0: Jeffrey4l_: need a bit of help. In kube when I start instance, I do not see that security group associated with instance gets programmed on linux bridge where the instance's tap interface plugged. WHich process is responsible for programming security group, is it neutron openvswitch agent or nova libvirt? | 14:16 |
| Daviey | sdake: sitrep? | 14:16 |
| sdake | Daviey we need to write teh pxeboot filee - how to do so? | 14:16 |
| zhubingbing | https://review.openstack.org/#/c/346448/ | 14:17 |
| zhubingbing | I will download it for debugging, if there is a problem I will be review | 14:17 |
| Jeffrey4l_ | sbezverk_, should be the neutron openvswitch. | 14:17 |
| Jeffrey4l_ | nova libvirt's security group only work with nova-network now. | 14:18 |
| sdake | why are we removing all of the cobbler systems? | 14:18 |
| Daviey | sdake: generate the files for tftp? | 14:18 |
| Daviey | sdake: cobbler sync | 14:19 |
| sdake | inc0 just run cobbler sync | 14:19 |
| sdake | inc0 rather then removing everything...........! | 14:19 |
| inc0 | I can generate it back in 10s | 14:19 |
| zhubingbing | ping egonzalez90 | 14:20 |
| Daviey | sdake: where have all the systems gone?! | 14:20 |
| sdake | inc0 removed them | 14:21 |
| sdake | no idea why | 14:21 |
| Daviey | Did anyone look at the iLo web based console? | 14:21 |
| sdake | probablly needs more cocaine bear imo | 14:21 |
| sdake | Daviey the os output nothing | 14:21 |
| Daviey | sdake: not the serial console, but the vga one? | 14:21 |
| sdake | Daviey - ya there is a screenshot from Jeffrey4l_ in scrollback | 14:22 |
| Daviey | sdake: directly after pxe? | 14:22 |
| *** neilus has joined #openstack-kolla | 14:22 | |
| Daviey | i see it | 14:23 |
| sbezverk_ | Jeffrey4l_: thank you.. will keep digging then in neutron openvswitch logs | 14:23 |
| sdake | the problem now is centos and kickstart configs are gone | 14:23 |
| inc0 | what? | 14:23 |
| inc0 | what is gone, everything is there. | 14:23 |
| sdake | i mean gone from the configuration of 23 | 14:24 |
| sdake | both the image and kernel command lines need to be set for 23 | 14:24 |
| sdake | and the kickstart file as well | 14:24 |
| Daviey | sdake: no, the issue is that we are missing the systems | 14:24 |
| sdake | Daviey - inc0 removed all the systems... | 14:25 |
| sdake | Daviey we can dd them back easily | 14:25 |
| sdake | Daviey the only system we care about atm is 23 | 14:25 |
| *** rkrum has joined #openstack-kolla | 14:25 | |
| inc0 | btw I removed all the *systems* not *profiles* | 14:26 |
| inc0 | sdake, we have a call with att now | 14:27 |
| sdake | serial console=tty0 console=ttyS0,115200n8 | 14:27 |
| sdake | rhallisey ping | 14:31 |
| *** banix_ has joined #openstack-kolla | 14:34 | |
| *** rhallisey has quit IRC | 14:35 | |
| *** banix has quit IRC | 14:36 | |
| *** banix_ is now known as banix | 14:36 | |
| sbezverk_ | sdake: I can access now the instance from outside :-) with floating ip | 14:37 |
| * Daviey needs to go again | 14:37 | |
| sbezverk_ | sdake: found a bug in neutron which was preventing it | 14:37 |
| *** rhallisey has joined #openstack-kolla | 14:41 | |
| *** lrensing__ has joined #openstack-kolla | 14:42 | |
| *** senk has joined #openstack-kolla | 14:44 | |
| *** dave-mcc_ has quit IRC | 14:49 | |
| *** huikang has quit IRC | 14:56 | |
| *** huikang has joined #openstack-kolla | 14:56 | |
| *** caowei has quit IRC | 14:56 | |
| *** mewald has quit IRC | 14:57 | |
| *** rhallisey has quit IRC | 15:03 | |
| *** srwilkers__ has joined #openstack-kolla | 15:03 | |
| *** rhallisey has joined #openstack-kolla | 15:04 | |
| sdake | sbezverk_ hot | 15:04 |
| *** banix has quit IRC | 15:04 | |
| *** banix has joined #openstack-kolla | 15:05 | |
| srwilkers__ | good morning everyone \o/. does anyone have an idea of something i could work on similar to the recent vmtp work? | 15:06 |
| zhubingbing | ping Mathias | 15:07 |
| sdake | there ae probbaly some big tent projects which can be integrated | 15:07 |
| sdake | just a moment - let me see | 15:07 |
| sdake | srwilkers__ we generaly go from the user surveyt o pick out features | 15:07 |
| srwilkers__ | cool, good to know | 15:08 |
| *** lrensing__ has quit IRC | 15:08 | |
| srwilkers__ | also makes sense :D | 15:08 |
| sdake | https://www.openstack.org/assets/survey/April-2016-User-Survey-Report.pdf | 15:08 |
| inc0 | tbh, I think there are better ways to learn than create new Dockerfile | 15:08 |
| inc0 | but that depends on how your brain operates | 15:08 |
| inc0 | I learn by trying stuff and fixing what's broken | 15:09 |
| sdake | srwilkers__ page 31 | 15:10 |
| sdake | srwilkers__ first you will need to hve a thoorough understanding of how aio works - have you got an aio deploy oging? | 15:10 |
| srwilkers__ | yeah | 15:11 |
| inc0 | then deploy ha multinode;) | 15:11 |
| *** huikang has quit IRC | 15:12 | |
| sdake | page 33 | 15:12 |
| *** huikang has joined #openstack-kolla | 15:12 | |
| *** signed8bit has joined #openstack-kolla | 15:12 | |
| sdake | rally seems similar in nature to vmtp | 15:12 |
| sdake | and has alot of community interest | 15:13 |
| srwilkers__ | okay cool | 15:13 |
| srwilkers__ | thanks guys :D | 15:13 |
| sdake | sahara trove | 15:13 |
| sdake | i think there isn't implementations for these yet | 15:13 |
| sdake | but htey maybe in queue | 15:13 |
| *** msimonin has quit IRC | 15:13 | |
| sdake | (the review quue) | 15:13 |
| srwilkers__ | ill take a gander at those | 15:14 |
| *** srwilkers_ has quit IRC | 15:14 | |
| sdake | make sur eto check that your not duplicating work already in flight | 15:14 |
| sdake | i am pretty sur enobody is working on rally | 15:14 |
| srwilkers__ | yeah, definitely | 15:14 |
| sdake | not sure about the rest on page 33 - i think most of those are in progress | 15:14 |
| *** dave-mccowan has joined #openstack-kolla | 15:14 | |
| *** michauds has joined #openstack-kolla | 15:15 | |
| inc0 | afk - relocationg:) | 15:16 |
| *** inc0 has quit IRC | 15:16 | |
| *** huikang has quit IRC | 15:17 | |
| *** sdake_ has joined #openstack-kolla | 15:18 | |
| Jeffrey4l_ | sdake, inc0, fyi, the test is done. you can start to re-install the OS | 15:18 |
| sdake_ | srs- read https://review.openstack.org/#/c/361253/2/doc/CONTRIBUTING.rst | 15:19 |
| sdake_ | Jeffrey4l_ how many iops we get? | 15:19 |
| sdake_ | Jeffrey4l_ can you amil me the html file of results ? | 15:19 |
| sdake_ | stdake@cisco.ccom | 15:19 |
| *** huikang has joined #openstack-kolla | 15:19 | |
| Jeffrey4l_ | i will push them to the PS. | 15:19 |
| Jeffrey4l_ | i will send it to u later. | 15:19 |
| sdake_ | i need it now though | 15:19 |
| sdake_ | where is it on disk - i'llcopy it off | 15:20 |
| Jeffrey4l_ | let's me check the iops. | 15:20 |
| Jeffrey4l_ | np | 15:20 |
| *** sdake has quit IRC | 15:20 | |
| sdake_ | scp the html to mybox if you can? | 15:20 |
| Jeffrey4l_ | ok | 15:21 |
| Jeffrey4l_ | sdake_, which folder? | 15:21 |
| *** rkrum has quit IRC | 15:23 | |
| *** vbel has quit IRC | 15:24 | |
| Jeffrey4l_ | sdake_, i put them here /home/jeffrey/testresult | 15:24 |
| *** vbel has joined #openstack-kolla | 15:24 | |
| *** sdake has joined #openstack-kolla | 15:24 | |
| Jeffrey4l_ | sdake, i put them here /home/jeffrey/testresult | 15:25 |
| *** dwalsh has joined #openstack-kolla | 15:25 | |
| sdake | Jeffrey4l_ what is the delta betwen the boot files? (.1, .2) | 15:26 |
| *** sdake_ has quit IRC | 15:26 | |
| *** zhubingbing has quit IRC | 15:26 | |
| Jeffrey4l_ | i tested twice using the the same rally tasks. i re-deployed openstack before .2. | 15:26 |
| Jeffrey4l_ | .1 has much failed tasks. | 15:27 |
| Jeffrey4l_ | .2 is more better. | 15:27 |
| *** vhosakot has joined #openstack-kolla | 15:27 | |
| sdake | 2 doesntshow iops just cumulative run time | 15:29 |
| Jeffrey4l_ | nova.boot test just boot instance test. | 15:29 |
| Jeffrey4l_ | for the io test. check the secnario11 folder | 15:29 |
| Jeffrey4l_ | i made two test: boot multi instances and io test | 15:30 |
| sdake | i did - html doesn't have iops printed out :) | 15:30 |
| sdake | Jeffrey4l_ ca nyou tell me how to interepet that data from scenario11 | 15:30 |
| Jeffrey4l_ | sdake, check the | 15:30 |
| Jeffrey4l_ | Scenario Data tab | 15:30 |
| sdake | Jeffrey4l_ i am there now | 15:32 |
| *** harlowja_at_home has joined #openstack-kolla | 15:32 | |
| sdake | 5796 iops? | 15:32 |
| sdake | is that the average ? | 15:32 |
| Jeffrey4l_ | sdake, the usage full data is bw and iops. | 15:32 |
| Jeffrey4l_ | sdake, which task are u checking? | 15:32 |
| Jeffrey4l_ | boot_runcommand_delete [?] | 15:33 |
| sdake | VMTasks.boot_runcommand_delete (164.512s) | 15:33 |
| sdake | which i assume is an average | 15:33 |
| Jeffrey4l_ | it do not calculate the sum iops. | 15:33 |
| Jeffrey4l_ | we should sum the ios when neccessary. | 15:34 |
| *** hieulq_ has joined #openstack-kolla | 15:35 | |
| Jeffrey4l_ | sdake, but you are checking the 1 node test. so the iops is the final value: 5795 | 15:35 |
| Jeffrey4l_ | for randwrite + 4K + 4 numjob + direct io + libaio engine | 15:36 |
| *** mewald has joined #openstack-kolla | 15:36 | |
| Jeffrey4l_ | other test may be using multi node, which we need sum all iops on each node. | 15:36 |
| *** zhubingbing_ has joined #openstack-kolla | 15:37 | |
| sdake | Jeffrey4l_ so tests 2-24 are multinode, tst 1 is single enode? | 15:37 |
| sdake | btw master upgrade is broken | 15:38 |
| sdake | if anyone cares to fix it ;-) | 15:38 |
| Jeffrey4l_ | sdake, no. I will such a test http://paste.openstack.org/show/564594/ | 15:39 |
| Jeffrey4l_ | there should be 4 1 node test. for each 4k rand write, 4k rand read, 4m rand write and 4m rand read. | 15:39 |
| Jeffrey4l_ | sdake, bug link? | 15:39 |
| sdake | Jeffrey4l_ didn't file bug | 15:40 |
| sdake | Jeffrey4l_ noticed last night when trying to time upgrade for master | 15:40 |
| Jeffrey4l_ | what about it is ? | 15:40 |
| sdake | upgrade from mitaka to master does not work - related to the bond interfaces not being fund | 15:40 |
| sdake | ansible variable error of some kind | 15:40 |
| Jeffrey4l_ | sdake, could u or any other file a bug with some log? | 15:41 |
| sdake | Jeffrey4l_ yes - if your done testing i'll do now | 15:41 |
| sdake | are ou running master or mitaka? | 15:42 |
| Jeffrey4l_ | no now. I need to bed, soon. I can test it tomorrow :) | 15:42 |
| sdake | Jeffrey4l_ we are redeplooying the cluster with centos | 15:42 |
| sdake | Jeffrey4l_ so if you want any tests nows the time to do em | 15:43 |
| Jeffrey4l_ | The total iops > 3w for 4k read write | 15:43 |
| sdake | i'll run a master upgrade and get a traceback | 15:43 |
| sdake | what is 3w? | 15:43 |
| Jeffrey4l_ | io test. the 4k rand write iops arouch 3w | 15:44 |
| Jeffrey4l_ | around. | 15:44 |
| sdake | what is w in that sentence? | 15:44 |
| sdake | you mean weeks? | 15:44 |
| Jeffrey4l_ | when u will destroy the current os? | 15:44 |
| Jeffrey4l_ | my bad.... the iops is around 30000 | 15:44 |
| sdake | as soon as we can figure out how to get centos installed on on enode | 15:44 |
| sdake | but we may not be able to figure that out | 15:45 |
| sdake | its not looking promising | 15:45 |
| sdake | since i dont see it happening on the gear | 15:45 |
| sdake | Jeffrey4l_ how did you calculate IOPS? | 15:45 |
| Jeffrey4l_ | OK. i will try another test. | 15:45 |
| Jeffrey4l_ | sdake, check this boot_runcommand_delete [6] in the html | 15:45 |
| Jeffrey4l_ | open the scenario data tab. | 15:46 |
| sdake | i've got that open | 15:46 |
| Jeffrey4l_ | check the fio: write graphic. click the iops on the top | 15:46 |
| Jeffrey4l_ | let is show the iops only. | 15:46 |
| Jeffrey4l_ | then you will found all the iops is above 1000 | 15:46 |
| Jeffrey4l_ | so 32 nodes * 1000 > 30000 | 15:47 |
| sdake | it was run on 32 nodes? | 15:48 |
| *** matrohon has quit IRC | 15:48 | |
| Jeffrey4l_ | yes. see the input task tab sdake | 15:48 |
| Jeffrey4l_ | concurrency: 32 times: 32 | 15:48 |
| sdake | cool | 15:48 |
| sdake | nice scenario teset jejff | 15:49 |
| sdake | Jeffrey4l_ | 15:49 |
| Jeffrey4l_ | but two nodes are failed. so should be 30 nodes * 10000 | 15:49 |
| Jeffrey4l_ | * 1000 | 15:49 |
| sdake | so 30k | 15:49 |
| Jeffrey4l_ | yep. | 15:49 |
| sdake | hotness | 15:49 |
| sdake | that is smoking fast | 15:49 |
| Jeffrey4l_ | are u saying it is good? or bad? | 15:50 |
| sdake | good | 15:50 |
| Jeffrey4l_ | :) | 15:50 |
| Jeffrey4l_ | i will continue the test: deploy ceph-osd on all node. then we will get a huge cluster of ceph. | 15:50 |
| Jeffrey4l_ | then test the performance. | 15:51 |
| sdake | Jeffrey4l_ daviey also wanted to do this same thing | 15:51 |
| Jeffrey4l_ | until u guys can deploy the centos . | 15:51 |
| Jeffrey4l_ | sdake, same thing mean? ceph-osd? | 15:51 |
| sdake | same scenario test | 15:51 |
| sdake | lots of osds | 15:51 |
| sdake | just deploy on compute - not control | 15:52 |
| Jeffrey4l_ | OK. I can deploy it now. | 15:52 |
| sdake | on compute and storage | 15:52 |
| Jeffrey4l_ | yes. | 15:52 |
| Jeffrey4l_ | got it. | 15:52 |
| Jeffrey4l_ | OK. start to deploy | 15:52 |
| sdake | note we have one less machine | 15:52 |
| sdake | one of them died | 15:52 |
| sdake | so I took it out of the inventoyr file | 15:52 |
| Jeffrey4l_ | I marked that. in the osic_playbook/inventory file | 15:53 |
| Jeffrey4l_ | hmm. which one? from when? | 15:53 |
| sdake | i dont remember | 15:54 |
| sdake | 140 i think | 15:54 |
| sdake | but it culd lhave been 40 | 15:54 |
| *** egonzalez90 has joined #openstack-kolla | 15:54 | |
| sdake | dunno Jeffrey4l_ - had 8 hours of sleep in last 5 days | 15:54 |
| sdake | i am beat | 15:54 |
| Jeffrey4l_ | OK should be it. i have comment it out. | 15:54 |
| *** neilus has quit IRC | 15:54 | |
| *** mark-casey has joined #openstack-kolla | 15:54 | |
| Jeffrey4l_ | sdake, take care :) | 15:55 |
| sdake | i'm doing what I can :) | 15:55 |
| Daviey | Jeffrey4l_: please do carry on | 15:55 |
| Daviey | i already made an inventory that folded storage into compute | 15:55 |
| *** zhurong has quit IRC | 15:55 | |
| *** apuimedo is now known as apuimedo|away | 15:56 | |
| Jeffrey4l_ | fold storage into compute? or vice versa? Daviey | 15:57 |
| Jeffrey4l_ | Daviey, start to destroy current openstack. OK? | 15:57 |
| sdake | Jeffrey4l_ yes go for it | 15:59 |
| sdake | Jeffrey4l_ quick q - how would i run that fio test on my own hardwre? | 15:59 |
| *** lyncos has quit IRC | 15:59 | |
| sdake | Jeffrey4l_ i'm interested in a bare metal baseline | 15:59 |
| Jeffrey4l_ | sdake, run fio directly? | 16:00 |
| sdake | yup | 16:00 |
| Jeffrey4l_ | you can get the fio command from the html | 16:00 |
| sdake | single node | 16:00 |
| sdake | ok - just run tht then? | 16:00 |
| Jeffrey4l_ | Runing fio by: ['fio', '-thread', '--output-format=json', '--group_reporting', '--filename', '/data', '--readwrite', 'randwrite', '--runtime', '120', '--size', '20G', '--iodepth', '64', '--ioengine', 'libaio', '--blocksize', '4k', '--numjobs', '4', '--name', 'fio test', '--direct', '1'] | 16:00 |
| *** neilus has joined #openstack-kolla | 16:00 | |
| Jeffrey4l_ | like this. | 16:00 |
| Jeffrey4l_ | yes. | 16:00 |
| Jeffrey4l_ | just notice the --filename , you may need change the value. | 16:01 |
| Jeffrey4l_ | install it from the epel-release directly. yum install fio | 16:01 |
| *** david-lyle_ is now known as david-lyle | 16:01 | |
| Daviey | Jeffrey4l_: sure, if you want | 16:01 |
| Jeffrey4l_ | Daviey, np. started to destroy and re-deploiy | 16:02 |
| Daviey | Jeffrey4l_: diff inventory-ips-only inventory-ips-only-storage-colo | 16:02 |
| *** hieulq_ has quit IRC | 16:03 | |
| Jeffrey4l_ | Daviey, you left the original storage nodes. | 16:04 |
| Daviey | Jeffrey4l_: hmm? | 16:04 |
| sdake | Jeffrey4l_ - any thoughtso n this? http://paste.fedoraproject.org/416959/72486655/ -> http://paste.fedoraproject.org/416959/72486655 | 16:04 |
| Daviey | Jeffrey4l_: I moved all the storage nodes into compute, and then made storage == compute | 16:04 |
| Jeffrey4l_ | Daviey, got it. | 16:04 |
| Jeffrey4l_ | You are right. I am doing almost the same things. | 16:05 |
| Daviey | cool | 16:05 |
| sdake | Jeffrey4l_ it appears fio isn't doing anything useful :) | 16:05 |
| Jeffrey4l_ | sdake, yep. you can remove the --output-format=json paramater, to generate a readable report. | 16:05 |
| sdake | it exits in about .5 seconds | 16:06 |
| Jeffrey4l_ | some wrong. | 16:06 |
| sdake | ya perm denied | 16:06 |
| sdake | got it with sudo ;) | 16:07 |
| Jeffrey4l_ | yep. remove the --output-format=json , too. | 16:07 |
| sdake | yup i did | 16:07 |
| sdake | this is my bare metal disk: | 16:09 |
| sdake | write: io=39090MB, bw=333455KB/s, iops=83363, runt=120041msec | 16:09 |
| *** zhubingbing_ has quit IRC | 16:09 | |
| Jeffrey4l_ | ssd | 16:09 |
| sdake | nvme ;) | 16:10 |
| Jeffrey4l_ | lOl | 16:10 |
| sdake | going to try on the teset gear | 16:10 |
| Jeffrey4l_ | write: io=2315.8MB, bw=162390KB/s, iops=40597, runt= 14598msec mine. ssd | 16:12 |
| *** daneyon has joined #openstack-kolla | 16:13 | |
| *** huikang has quit IRC | 16:16 | |
| *** huikang has joined #openstack-kolla | 16:17 | |
| *** daneyon has quit IRC | 16:17 | |
| *** sdake has quit IRC | 16:19 | |
| *** huikang has quit IRC | 16:21 | |
| *** neilus has quit IRC | 16:22 | |
| *** neilus has joined #openstack-kolla | 16:22 | |
| *** inc0 has joined #openstack-kolla | 16:23 | |
| *** huikang has joined #openstack-kolla | 16:24 | |
| *** salv-orlando has quit IRC | 16:27 | |
| *** salv-orlando has joined #openstack-kolla | 16:27 | |
| *** huikang has quit IRC | 16:28 | |
| *** sdake has joined #openstack-kolla | 16:30 | |
| *** salv-orlando has quit IRC | 16:30 | |
| sdake | Jeffrey4l_ where are the ssd's on 253? | 16:32 |
| Jeffrey4l_ | sdake, no idea. :( | 16:35 |
| sdake | i need a baseline on the ssds for fio | 16:36 |
| *** aNupoisc has joined #openstack-kolla | 16:36 | |
| sdake | is there one i can use? | 16:36 |
| Jeffrey4l_ | you can use 0.23 for the test. | 16:38 |
| *** harlowja_at_home has quit IRC | 16:39 | |
| sdake | no we nuked 23 | 16:40 |
| sdake | we are using 124 nodes, 2 are down that should leave 5 extra nodes that are around | 16:40 |
| sdake | have you started your test yet on the other nodes? | 16:40 |
| Jeffrey4l_ | sdake, try this ssh root@172.21.0.66 | 16:41 |
| sdake | that works thanks | 16:42 |
| Jeffrey4l_ | in inventory file, it say: DNS broken for some reason | 16:43 |
| *** unicell has quit IRC | 16:43 | |
| sdake | Jeffrey4l_ in all of our tests, /var/lib/docker is not on an ssd... | 16:47 |
| Daviey | umm | 16:48 |
| Daviey | sdake: I thought all disks were ssd on these? | 16:48 |
| sdake | i ont think /dev/sda is | 16:48 |
| sdake | i dont really know | 16:48 |
| sdake | perhaps it is | 16:48 |
| sdake | so here is ssd baseline in osic: | 16:48 |
| sdake | write: io=18417MB, bw=157153KB/s, iops=39288, runt=120004msec | 16:48 |
| sdake | my workstations do almost double iops :) | 16:49 |
| Jeffrey4l_ | sdake, that's OK. we write data to ceph/ssd directly | 16:49 |
| sdake | Jeffrey4l_ i know but data volumes are not on ceph | 16:49 |
| sdake | and ddepoy would surelygo faster to ssd | 16:50 |
| sdake | butI think sda is an ssd - going to find out | 16:50 |
| Jeffrey4l_ | sdake, few thing are wrote in the volumes. | 16:50 |
| sdake | [ 2.266417] sd 0:0:0:0: [sda] 1562758832 512-byte logical blocks: (800 GB/745 GiB) | 16:50 |
| sdake | all the diks are the same type | 16:51 |
| Jeffrey4l_ | inc0, just mount a large volume to /var/lib/docker | 16:51 |
| inc0 | yeah | 16:51 |
| inc0 | we now have 600+gigs for elastic \o/ | 16:52 |
| mark-casey | lol, so. two weeks :D | 16:52 |
| Jeffrey4l_ | i disabled the debug log. so the log will not grow that much. | 16:52 |
| sdake | i looekd into gcing es | 16:52 |
| sdake | its super easy | 16:52 |
| sdake | Jeffrey4l_ debug was on this entire time? | 16:53 |
| Jeffrey4l_ | sdake, seems yes. :( | 16:53 |
| *** gfhellma has joined #openstack-kolla | 16:53 | |
| Jeffrey4l_ | I disabled it today. | 16:53 |
| sdake | thats ok - most people want to run their cluster with debug | 16:53 |
| jrich523 | how many more days left with the cluster? | 16:57 |
| jrich523 | 2 or 3 right? | 16:57 |
| inc0 | I haven't seen ceph with that many OSDs before | 16:58 |
| inc0 | gonna be interesting | 16:58 |
| *** gfidente has quit IRC | 16:59 | |
| sdake | 1 | 16:59 |
| sdake | 1 | 16:59 |
| Jeffrey4l_ | more osd means more fast ( i think) | 17:00 |
| inc0 | not necessarly | 17:00 |
| inc0 | mons can be overwhelmed | 17:00 |
| inc0 | also wont really affect speed | 17:00 |
| Jeffrey4l_ | maybe . | 17:00 |
| inc0 | network for replication can become issue as well, if we save a lot of data there | 17:01 |
| inc0 | this will be huge cluster as ceph goes | 17:01 |
| inc0 | well, biggest one I've ever seen | 17:01 |
| Jeffrey4l_ | me too. :) | 17:01 |
| *** lyncos has joined #openstack-kolla | 17:02 | |
| lyncos | 3GHZ | 17:02 |
| Jeffrey4l_ | 3 mon + 110 * 10 nodes | 17:02 |
| inc0 | hey lyncos :) | 17:02 |
| lyncos | Hey guys | 17:02 |
| Jeffrey4l_ | 3 mon + 110 * 10 disks ( 1100+ osd ) | 17:02 |
| inc0 | 1k+ OSDs | 17:02 |
| inc0 | gonna be fun | 17:02 |
| lyncos | You all miss me | 17:02 |
| inc0 | I want to get ceph -s after this | 17:03 |
| jrich523 | i want to get a tesla s after this | 17:04 |
| jrich523 | oh wait.... completely different set of desires | 17:04 |
| inc0 | we need to up placement groups....a lot | 17:04 |
| Jeffrey4l_ | inc0, yes. | 17:04 |
| inc0 | Jeffrey4l_, clock skew detected | 17:05 |
| inc0 | did you install ntp? | 17:05 |
| Jeffrey4l_ | inc0, no.. | 17:05 |
| Jeffrey4l_ | let's install. | 17:05 |
| Jeffrey4l_ | inc0, we need 40K pg in sum | 17:11 |
| inc0 | nearest power of 2 | 17:12 |
| inc0 | of this | 17:12 |
| Jeffrey4l_ | i got this | 17:13 |
| Jeffrey4l_ | PG num = (OSDs * 100/max_rep_count) | 17:13 |
| Jeffrey4l_ | PG per pool = (PG num)/NB_pools | 17:13 |
| inc0 | kk | 17:13 |
| Jeffrey4l_ | so the pg num should be 1200 * 100 /3 = 40K | 17:13 |
| inc0 | this thrown errors | 17:13 |
| inc0 | ahh you added different pools? | 17:14 |
| Jeffrey4l_ | inc0, yeah. | 17:14 |
| inc0 | ok | 17:14 |
| Jeffrey4l_ | inc0, increate little by little | 17:14 |
| sdake | so are we giving up on centos then? | 17:15 |
| inc0 | so it seems | 17:15 |
| Jeffrey4l_ | set vms to 40K? no need touch other pools. | 17:15 |
| inc0 | yeah go ahead | 17:16 |
| Jeffrey4l_ | ok | 17:16 |
| sdake | inc0 i'd really like a centos baseline | 17:16 |
| sdake | atleast one test case | 17:16 |
| sdake | is it not possiblee to eeploy centos within the neext 12 hours? | 17:17 |
| *** huikang has joined #openstack-kolla | 17:17 | |
| sdake | inc0 ^^ | 17:19 |
| *** kangh_ has joined #openstack-kolla | 17:20 | |
| *** huikang has quit IRC | 17:23 | |
| *** dwalsh has quit IRC | 17:27 | |
| sdake | inc0 tell me now - so i can either stay and help or go back to bed | 17:27 |
| inc0 | sdake, I don't think it's feasable | 17:29 |
| inc0 | I would need to learn this stuff | 17:30 |
| inc0 | and I'm not good at centos | 17:30 |
| vhosakot | are there instructions? I'll help deploy centos | 17:31 |
| sdake | instructions | 17:32 |
| sdake | lol | 17:32 |
| sdake | if there were instructions it woud already be done | 17:32 |
| sdake | the only person i know that knows this stuff well is britthou_ | 17:33 |
| mark-casey | this is the sort of thing that turns into one rabbit hole after the other. If there is some way to segment (even by manual firewalling or etc.) 1 or 2 nodes and have a few people work on it then you could keep at the ceph stuff in the mean time. | 17:33 |
| *** unicell has joined #openstack-kolla | 17:33 | |
| mark-casey | That or accept the risk that you put 14 hours into it and get nothing further done on the cluster | 17:34 |
| sdake | we have a node segmented to deploy centos to | 17:34 |
| mark-casey | oh :D | 17:34 |
| sdake | so there is no risk of not getting anything furhter done | 17:34 |
| vhosakot | britthou_: ping | 17:35 |
| vhosakot | sdake: "node segmented to deploy centos to" means there is a PXE server to load centos onto all the target nodes ? | 17:35 |
| sdake | vhosakot no it means we have one node (0.23) to load centos onto | 17:36 |
| sdake | via cobbler | 17:36 |
| vhosakot | ah ok | 17:36 |
| sdake | then wecan expand to every otherndoe | 17:36 |
| sdake | if we can get one node working | 17:36 |
| *** wally__ has joined #openstack-kolla | 17:36 | |
| vhosakot | ah got it... it is the cobbler node | 17:36 |
| *** berendt has quit IRC | 17:36 | |
| wally__ | hi inc0,, r u there | 17:36 |
| *** wally__ is now known as Guest14981 | 17:36 | |
| Jeffrey4l_ | the fio test is started and will be finished in 2 hours | 17:37 |
| vhosakot | if we plan to do centos, I can help whoever is doing it | 17:37 |
| *** Guest14981 has quit IRC | 17:37 | |
| sdake | vhosakot inc0 seems to know th most about cobbler | 17:38 |
| sdake | vhosakot but hwere we er jammed up is in a kickstart file I think | 17:38 |
| sdake | but I'm not relaly sure | 17:38 |
| sdake | it seems like everyonegve up - nobody to lead the rest of the effort | 17:38 |
| *** lyncos has quit IRC | 17:38 | |
| sdake | I've got nothing left - selpt at most 8 hours since wednesday | 17:39 |
| *** wally1 has joined #openstack-kolla | 17:39 | |
| sdake | AND Its mONDAY | 17:39 |
| *** lyncos has joined #openstack-kolla | 17:39 | |
| sdake | so if someone sets it up, grand | 17:39 |
| wally1 | hi inc0. there | 17:39 |
| sdake | when i wake up i'll do a baseline | 17:39 |
| vhosakot | sdake: cool.. I'll check with inc0 | 17:40 |
| *** mewald has quit IRC | 17:40 | |
| sdake | vhosakot inc0 gave up - look at britthou_ - he cn get you going | 17:40 |
| vhosakot | sdake: cool... I'll check with britthou_.. about cobbler kickstarts and PXE... | 17:41 |
| vhosakot | sdake: when do you plan to finish the baseline on Ubuntu ? when is the cluster free for cenot re-image? | 17:42 |
| vhosakot | sdake: also, until when do we have the cluster for us ? end of day tomorrow, or end of day 31st ? | 17:42 |
| *** fragatina has quit IRC | 17:44 | |
| inc0 | sdake, did you look at pxe we produced? | 17:44 |
| vhosakot | inc0: do we need to create the centos kickstarts for cobbler ? | 17:46 |
| inc0 | vhosakot, yeah, I did something | 17:47 |
| inc0 | like copy pasted from inet | 17:47 |
| vhosakot | inc0: where is it.. I dont see it in tmux.. is it on the deploy node 002 ? | 17:47 |
| inc0 | but as I said - I'm not centos guy, and not really a cobbler guy either | 17:47 |
| inc0 | join session 9 in tmux | 17:47 |
| vhosakot | ah, session 9.. ok | 17:48 |
| inc0 | let me know when you're in | 17:48 |
| vhosakot | yep.. 2 mins | 17:48 |
| vhosakot | inc0: I'm in | 17:57 |
| wally1 | hi inc0, where r u | 17:58 |
| inc0 | I'm here now wally1 | 18:00 |
| inc0 | but a bit busy atm | 18:00 |
| *** lyncos has quit IRC | 18:01 | |
| *** fragatina has joined #openstack-kolla | 18:02 | |
| Jeffrey4l_ | need go bed. bye guys. | 18:05 |
| vhosakot | inc0: Jeffrey4l_ showed me the config | 18:05 |
| vhosakot | Jeffrey4l_: good night | 18:05 |
| vhosakot | inc0: how can I access the cobbler node (0.32) ? | 18:05 |
| vhosakot | 0.23 I meant | 18:06 |
| *** rmart04 has joined #openstack-kolla | 18:06 | |
| Pavo | centos is pretty easy, never used cobbler though, but I do use ks files to install centos using dnsmaq | 18:06 |
| Pavo | thats all we use is centos | 18:07 |
| *** lyncos has joined #openstack-kolla | 18:08 | |
| *** egonzalez90 has quit IRC | 18:08 | |
| *** rmart04 has quit IRC | 18:10 | |
| *** huikang has joined #openstack-kolla | 18:11 | |
| wally1 | .... hi inc0 there? | 18:11 |
| sdake | wally1 what do you need | 18:12 |
| vhosakot | inc0: thanks for the pointer.. I'll load it onto 0.23 and let you know | 18:12 |
| sdake | 0.23 is where centos should be loaded to | 18:12 |
| wally1 | last friday, I asked inc0 help for deploy the kolla all in one, I repare a server for him, he said he can helo me to instlal | 18:13 |
| vhosakot | sdake: yes, I'll load centos on 0.23 as a test.. and if it works, we can use the same ks to load centos on all the 130 nodes.. this is what inc0 told | 18:13 |
| wally1 | since I cannot get my workinng, | 18:13 |
| wally1 | the VM cannnot go out | 18:13 |
| inc0 | sdake, any chances you could try to help wally in my stead? | 18:13 |
| sdake | vhosakot cool soundsgood | 18:13 |
| inc0 | I'll stick with osic cluster | 18:13 |
| sdake | inc0 what do you think I'm doing :) | 18:13 |
| sdake | i culdn't sleep becausei just woke up | 18:14 |
| inc0 | you're the man sdake | 18:14 |
| sdake | need to take some mad sleep meds :) | 18:14 |
| vhosakot | inc0: I need the interface connected to the cobbler server, MAC address, IP address for the "cobbler system edit" command | 18:14 |
| sdake | wally1 tell me what type of serveroy uhae | 18:14 |
| inc0 | vhosakot, once you setup the profile | 18:14 |
| wally1 | ubunt 1404 servers | 18:14 |
| wally1 | i have the server reade | 18:14 |
| sdake | more then one? | 18:14 |
| inc0 | let me know, I'll show you how to cobller it up | 18:14 |
| wally1 | all in one | 18:14 |
| wally1 | i havef the server reazdy with public ip | 18:15 |
| sdake | multiple nics? | 18:15 |
| vhosakot | inc0: profile ? meaning kickstart, right ? | 18:15 |
| wally1 | i have 2 config | 18:15 |
| *** mewald has joined #openstack-kolla | 18:15 | |
| vhosakot | inc0: ah got it.. thanks | 18:15 |
| sdake | you mean 2 nics? | 18:15 |
| wally1 | yeah, | 18:15 |
| sdake | ok - follow quick start guide for evaluation/deployment | 18:16 |
| *** huikang has quit IRC | 18:16 | |
| sdake | you have been through it once | 18:16 |
| sdake | should be straightforward to run thorugh again | 18:16 |
| vhosakot | inc0: the profile points to ubuntu seed... I'll first update that | 18:16 |
| wally1 | i have install so many times, but nothing worikking | 18:16 |
| inc0 | yeah | 18:16 |
| wally1 | the VM canot go out side | 18:16 |
| sdake | ya - tht is because your vm is not setup properly | 18:16 |
| sdake | kolla can definately network outside | 18:16 |
| wally1 | that is why in0 said he can help me to depploy one for me | 18:16 |
| wally1 | :( | 18:16 |
| wally1 | do u have time to insttall one for me? | 18:17 |
| sdake | so go ahead and get to the point where you would deploy on the baremetal from the quickstart | 18:17 |
| sdake | yup i hae time | 18:17 |
| sdake | but wouldn't you rather do it? | 18:17 |
| vhosakot | inc0: I dont see any centos preseends in /opt/osic/preseeds | 18:17 |
| wally1 | i have following everything step in the guide | 18:17 |
| sdake | wally1 on the baremetal node? | 18:18 |
| wally1 | but no idea why not working | 18:18 |
| wally1 | yeah | 18:18 |
| wally1 | physical server | 18:18 |
| wally1 | dell server | 18:18 |
| sdake | what specifically isn't working? | 18:18 |
| sdake | does nova list work? | 18:18 |
| *** kangh__ has joined #openstack-kolla | 18:18 | |
| wally1 | the VM cannnot reacch outside, cannot ping and cannot ssh into thte VM | 18:18 |
| wally1 | yeak | 18:18 |
| sdake | wally1 did you run init-runonce? | 18:18 |
| wally1 | nova agent list working | 18:18 |
| wally1 | yeah, | 18:19 |
| wally1 | i did | 18:19 |
| sdake | ok - init-runonce is for my personal home lab environment | 18:19 |
| sdake | init-runoonce is not a generic tool for end user usage | 18:19 |
| wally1 | i change the ip in the innit-runonce | 18:19 |
| sdake | its meant to serve as a guide to get going | 18:19 |
| wally1 | that is why, can you help me to deploy all in one on my server, than i can take a look | 18:19 |
| wally1 | if you haveg time, | 18:20 |
| wally1 | only 15 - 20 mins right | 18:20 |
| sdake | yup - hae you gone throug hthe uickstart on your bare metal? | 18:20 |
| vhosakot | I don't think I can re-use an ubuntu seed for centos.. we'll need a new centos seed with all the YUM packages.. the ubuntu seed in debian-based | 18:20 |
| wally1 | yeah, I have followdd the link and do many times | 18:20 |
| *** kangh_ has quit IRC | 18:20 | |
| sdake | vhosakot i was pretty sure a preseed was an ubuntu/debian only thing but i could be wrong | 18:20 |
| sdake | wally1 are you able to connect to webex? | 18:21 |
| vhosakot | sdake: ah ok... seed is ubuntu's kickstart? inc0 showed me the centos ks.. | 18:22 |
| sdake | yup | 18:22 |
| *** mewald has quit IRC | 18:22 | |
| vhosakot | cool.. got it.. I'll point the cobbler profile to the centos ks | 18:22 |
| Daviey | uhm, what is the situation? | 18:25 |
| Daviey | We had a kickstart for centos already, but it wasn't functional | 18:25 |
| sdake | Daviey - cluster running for 2 hours a massive ceph test | 18:25 |
| sdake | ceph io test | 18:25 |
| sdake | also trying to boot .23 | 18:25 |
| Daviey | sdake: yeah, saw that. looking forward to benchmarks | 18:25 |
| sdake | Daviey th ekickstart for .23 was not set | 18:26 |
| Daviey | sdake: are we also loading it with sporadic nova vm's? | 18:26 |
| sdake | vhosakot you might have mor eluck if you turn on serial output | 18:26 |
| sdake | Daviey i dont know mechanics of the test case | 18:26 |
| vhosakot | sdake: serial output where ? how ? | 18:26 |
| sdake | vhosakot no idea | 18:26 |
| Daviey | sdake: erm, there was a kickstart set.. that is what i set before i went to bed | 18:26 |
| openstackgerrit | Ryan Hallisey proposed openstack/kolla-kubernetes: Update the kubernetes aio guide https://review.openstack.org/361787 | 18:27 |
| openstackgerrit | Ryan Hallisey proposed openstack/kolla-kubernetes: Add a script that will get the latest kubectl https://review.openstack.org/362312 | 18:27 |
| sdake | https://cisco.webex.com/ciscosales/e.php?MTID=m659c63768c3e0e10eeed6121bc1add01 | 18:27 |
| vhosakot | sdake: you mean output from the PXE port incase PXE loading crashes.. | 18:27 |
| sdake | wrong link | 18:27 |
| sdake | not even sure what that is | 18:27 |
| vhosakot | sdake: yes, we'll need serial access to debug boot crashes as the box wont even have an IP address to SSH into | 18:27 |
| sdake | vhosakot serial console=tty0 console=ttyS0,115200n8 | 18:27 |
| Daviey | vhosakot: we have IPMI serial console, and iLo VGA | 18:27 |
| sdake | vhosakot that needs to go in the kernel command line in some way | 18:28 |
| rhallisey | v1k0d3n, ^ just pushed some aio docs changes | 18:28 |
| rhallisey | those should be far more straightforward | 18:28 |
| sdake | although nobody but Jeffrey4l_ has been abble to get the ilo vgat o work | 18:28 |
| vhosakot | sdake: got it.. "serial console=tty0 console=ttyS0,115200n8" musy be added as kernel option when booting.. cool.. I'll add this in the centos ks | 18:28 |
| *** berendt has joined #openstack-kolla | 18:28 | |
| vhosakot | thanks Daviey | 18:29 |
| *** berendt has quit IRC | 18:29 | |
| vhosakot | sdake: the cobbler profile already has it --> Kernel Options : {'serial': '~', 'console': ['tty0', 'ttyS0,115200n8']} | 18:29 |
| sdake | ya i set it | 18:29 |
| sdake | not sure if i st it correctly | 18:30 |
| sdake | Daviey the kickstart was instead pointing to a prseed | 18:31 |
| sdake | anyway inc0 deleted all the sytems and strted with uust 0.23 | 18:31 |
| sdake | wally1 -> see privat emessage with webex location | 18:33 |
| *** salv-orlando has joined #openstack-kolla | 18:34 | |
| *** lrensing_ has quit IRC | 18:35 | |
| Daviey | sdake: it wasn't... | 18:37 |
| Daviey | i checked it was kickstart before i went to bed. :/ | 18:38 |
| Daviey | oh well | 18:38 |
| *** gfhellma has quit IRC | 18:39 | |
| wally1 | hi sdake | 18:39 |
| wally1 | sorry, just went out | 18:39 |
| wally1 | how do see private message with webex? | 18:39 |
| sdake | on irc i sent you a privat emessage | 18:40 |
| sdake | webex is a paltform where i can see your screen | 18:40 |
| sdake | on your server | 18:40 |
| wally1 | oh, do i need to install this? | 18:40 |
| wally1 | never use it before | 18:41 |
| *** salv-orlando has quit IRC | 18:42 | |
| vhosakot | I see we can use "cobbler profile edit --kickstart=" to update the ubuntu seed to centos ks.. I'll try it now | 18:44 |
| *** mnasiadka has quit IRC | 18:45 | |
| sdake | wally1 just click the link | 18:46 |
| sdake | wally1 http://cisco.webex.com/join/stdake | 18:46 |
| sdake | ok let me get my headset | 18:47 |
| sdake | moment | 18:47 |
| wally1 | yeah, just did | 18:47 |
| wally1 | did u see my screen | 18:48 |
| sdake | yup | 18:48 |
| wally1 | really | 18:49 |
| wally1 | so ... what to do now | 18:49 |
| sdake | wally1 click "Quickstart" tab and then click "connect to audio" | 18:49 |
| sdake | wally1 i just got my headset - hang tight - you need to get on audio so i can speak with you | 18:49 |
| wally1 | i have my headset | 18:50 |
| wally1 | too | 18:50 |
| wally1 | where is teh quickstart tab | 18:50 |
| sdake | on the top o f the screen | 18:50 |
| sdake | try stop sharing | 18:51 |
| sdake | or exit full screen mode | 18:51 |
| sdake | or exit entirely -don't share screen and click "connect to audio" | 18:51 |
| sdake | wally big blue button on left of screen says "connect to audio" | 18:52 |
| sdake | i use my compute rfor this function | 18:53 |
| sdake | ory ouc an dial in | 18:53 |
| *** daneyon has joined #openstack-kolla | 18:55 | |
| *** daneyon has quit IRC | 18:59 | |
| openstackgerrit | Ryan Hallisey proposed openstack/kolla-kubernetes: Add a script that will get the latest kubectl https://review.openstack.org/362312 | 19:00 |
| openstackgerrit | Ryan Hallisey proposed openstack/kolla-kubernetes: Update the kubernetes aio guide https://review.openstack.org/361787 | 19:00 |
| *** imcsk8 has joined #openstack-kolla | 19:03 | |
| openstackgerrit | Kevin Fox proposed openstack/kolla-kubernetes: Deployment, Readiness, Safe Shutdown, & Scaling for glance https://review.openstack.org/354895 | 19:04 |
| openstackgerrit | Ryan Hallisey proposed openstack/kolla-kubernetes: Update the quickstart https://review.openstack.org/362325 | 19:06 |
| *** huikang has joined #openstack-kolla | 19:12 | |
| openstackgerrit | Ryan Hallisey proposed openstack/kolla-kubernetes: Update the quickstart https://review.openstack.org/362325 | 19:13 |
| *** zhugaoxiao has quit IRC | 19:16 | |
| *** zhugaoxiao has joined #openstack-kolla | 19:16 | |
| wally1 | hi sdave thanks for your help | 19:16 |
| wally1 | i will re-configure my network | 19:17 |
| *** huikang has quit IRC | 19:17 | |
| wally1 | but one last question, if i change my kolla_internal_vip_address: to be "192.168.X.XX", then I cannot loign to my horizon dashboard from outside. | 19:18 |
| wally1 | but you said I can set the external ip for the horizion dashboard? right | 19:18 |
| wally1 | where in the golabl;ymal file | 19:19 |
| *** gfhellma has joined #openstack-kolla | 19:20 | |
| sdake | wally1 its clled kolla_external_vip_address | 19:24 |
| *** dave-mccowan has quit IRC | 19:25 | |
| vhosakot | inc0: ping | 19:29 |
| inc0 | I'm here | 19:29 |
| vhosakot | I ran "cobbler profile edit --name Centos-7-x86_64 --kickstart=/var/lib/cobbler/kickstarts/centos7.ks" | 19:29 |
| vhosakot | but only one ks got changed | 19:29 |
| vhosakot | I still see some ubuntu seeds | 19:29 |
| vhosakot | I see we can do "cobbler profile getks" to get the centos ks and load it onto 0.23 | 19:30 |
| inc0 | vhosakot, tbh...I dont know how to cobbler:/ | 19:30 |
| openstackgerrit | Merged openstack/kolla: Fix removing nova_libvirt container issue https://review.openstack.org/361728 | 19:30 |
| vhosakot | inc0: how was the Centos-7-x86_64 profile created in the first place ? was it copied from the ubuntu profile using "cobbler profile copy" ? | 19:31 |
| inc0 | tbh I don't know | 19:31 |
| inc0 | wasn't me | 19:31 |
| sdake | Daviey created that profile | 19:31 |
| vhosakot | sdake: cool.. Daviey: did you copy an Ubuntu profile and made the Centos-7-x86_64 profile.. or, you created Centos-7-x86_64 profile from scratch using "cobbler profile add" ? | 19:32 |
| sdake | vhosakot don't recall what ommand he used | 19:32 |
| openstackgerrit | Michal Jastrzebski (inc0) proposed openstack/kolla: Customization for base https://review.openstack.org/358042 | 19:33 |
| vhosakot | cool | 19:33 |
| Daviey | vhosakot: that is created automagically when you create a distro | 19:33 |
| vhosakot | Daviey: the Centos-7-x86_64 profile points to ubuntu seeds and I'm not able to change it | 19:33 |
| Daviey | er | 19:33 |
| vhosakot | Daviey: http://paste.openstack.org/show/564642/.. see the line with <----- | 19:34 |
| *** huikang has joined #openstack-kolla | 19:34 | |
| vhosakot | I was able to change only one ks of the Centos-7-x86_64 profile using the command "cobbler profile edit --name Centos-7-x86_64 --kickstart=/var/lib/cobbler/kickstarts/centos7.ks" but not all ks | 19:34 |
| Daviey | vhosakot: that is available profiles | 19:34 |
| vhosakot | Daviey: it is the report from just Centos-7-x86_64 , right ? I ran "cobbler profile report Centos-7-x86_64" | 19:35 |
| sdake | try with --name Centos-7-x86_64 | 19:36 |
| vhosakot | sdake: yes.. passing --name shows the right output! looks like the centos profile does point to the centos ks.. | 19:37 |
| vhosakot | cobbler profile report --name Centos-7-x86_64 | grep -i kick | 19:37 |
| vhosakot | Kickstart : /var/lib/cobbler/kickstarts/centos7.ks | 19:37 |
| vhosakot | Kickstart Metadata : {} | 19:37 |
| vhosakot | what is the next step? "cobbler profile getks" to generate the centos ks ? | 19:37 |
| sdake | i need to feed my brain | 19:37 |
| sdake | i'll be back | 19:37 |
| vhosakot | I mean "cobbler profile getks --name Centos-7-x86_64" | 19:38 |
| vhosakot | sdake: cool... later | 19:38 |
| Daviey | vhosakot: see the screen? | 19:38 |
| Daviey | vhosakot: this is a distraction, you can see the kickstart there | 19:38 |
| *** fragatina has quit IRC | 19:39 | |
| Daviey | That isn't the issue | 19:39 |
| vhosakot | Daviey: which screen? I'm in window 4 in tmux on the osic-prep machine | 19:39 |
| v1k0d3n | rhallisey: aio for kolla-kube? man...was just going to try and work on that :) | 19:39 |
| *** fragatina has joined #openstack-kolla | 19:39 | |
| rhallisey | v1k0d3n, I had a patch up for it last night :) | 19:39 |
| rhallisey | just tweaked it a little this morning | 19:39 |
| rhallisey | v1k0d3n, see if you can get the vagrant stuff from kargo working | 19:40 |
| Daviey | vhosakot: tmux session 0 | 19:40 |
| rhallisey | It didn't work for me | 19:40 |
| v1k0d3n | awesome. | 19:41 |
| vhosakot | Daviey: I'm in tmux window 0 | 19:41 |
| vhosakot | do you see me? I typed" #vhosakot here" | 19:41 |
| Daviey | nope | 19:41 |
| Daviey | vhosakot: are you on tmux session 0, window 0? | 19:41 |
| *** salv-orlando has joined #openstack-kolla | 19:42 | |
| vhosakot | window 0 | 19:42 |
| vhosakot | Daviey: what command should I use to go to tmux session 0? | 19:42 |
| Daviey | ssh, tmux att -t 0 | 19:42 |
| vhosakot | Daviey: I'm in | 19:43 |
| *** salv-orlando has quit IRC | 19:44 | |
| openstackgerrit | Michal Jastrzebski (inc0) proposed openstack/kolla: Architecture guide doc https://review.openstack.org/360796 | 19:46 |
| *** dave-mccowan has joined #openstack-kolla | 19:47 | |
| *** sdake has quit IRC | 19:48 | |
| *** wally1 has quit IRC | 19:50 | |
| *** aswadr_ has quit IRC | 19:51 | |
| Daviey | vhosakot: are you conneted to iLo vga? | 19:52 |
| *** huikang has quit IRC | 19:54 | |
| Daviey | vhosakot: ?? | 19:56 |
| vhosakot | Daviey: yes, I'm in tmux session 0 | 19:57 |
| mark-casey | hey all any results on ceph yet? curious if the osic hardware puts up some good numbers | 19:58 |
| Daviey | vhosakot: hey, so... serial console is great... but you don't always get all messages - or the option to really play with the bios | 19:59 |
| Daviey | VGA iLO is pretty good to use aswell | 20:00 |
| Daviey | .net windows support works well.. Java somewhat. | 20:00 |
| vhosakot | Daviey: cool | 20:00 |
| Daviey | But, i am a linux desktop person.. which really gives me the shitty end of the stick on these | 20:00 |
| Daviey | vhosakot: so we have successful install | 20:01 |
| vhosakot | Daviey: so, once the boot process in the bottom pane finishes, will we see the login prompt? | 20:01 |
| vhosakot | Daviey: why is the test target machine called 0.23? it the IP it gets x.y.0.23 ? | 20:02 |
| *** inc0 has quit IRC | 20:02 | |
| vhosakot | Daviey: got it.. I see its IP is 172.21.0.23.. cool | 20:03 |
| Daviey | right | 20:04 |
| vhosakot | looks like it booted but password is not working? | 20:04 |
| Daviey | vhosakot: The password was badly set | 20:04 |
| *** inc0 has joined #openstack-kolla | 20:04 | |
| Daviey | someone set it to --iscrypted cobbler | 20:04 |
| Daviey | which means cobbler is the result of the salting | 20:04 |
| Daviey | (and i'm guessing that isn't true :) | 20:04 |
| vhosakot | Daviey: so, it means, it wont accept plaintext password from a keyboard ? | 20:04 |
| Daviey | vhosakot: it means i have NFI what the password is | 20:04 |
| vhosakot | Daviey: ah ok :) | 20:06 |
| mark-casey | you guys should use the other nodes to crack it | 20:06 |
| vhosakot | can we change it to a good password, and re-run the ks again? | 20:06 |
| *** huikang has joined #openstack-kolla | 20:08 | |
| Daviey | vhosakot: done | 20:09 |
| vhosakot | Daviey: cool.. we'll see how it boots this time.. | 20:09 |
| vhosakot | so, the cobbelr hirarchy is system --> profile --> ks --> image... is this right ? | 20:10 |
| Daviey | vhosakot: think so | 20:10 |
| Daviey | (I hate cobbler) | 20:10 |
| vhosakot | there is PXE and DHCP as well in that hierarchy somewhere | 20:12 |
| openstackgerrit | Merged openstack/kolla-kubernetes: Getting NOVA pieces together to launch VM https://review.openstack.org/358911 | 20:14 |
| Daviey | vhosakot: well yes, but that isn't inherent to cobbler | 20:14 |
| Daviey | there is a dhcp server, which has dhcp option 66 pointing to the cobbler servers tftp server | 20:15 |
| Daviey | in /var/lib/tftp/ there is the tftp files that the bios pulls down | 20:15 |
| vhosakot | Daviey: ah ok cool | 20:15 |
| Daviey | they are created by cobbler | 20:15 |
| vhosakot | Daviey: are the steps you run documented anywhere ? | 20:17 |
| Daviey | vhosakot: don't think so | 20:18 |
| *** huikang_ has joined #openstack-kolla | 20:22 | |
| vhosakot | Daviey: I see the boo process in the bottom pane empty/ended ? | 20:23 |
| vhosakot | boot* | 20:23 |
| vhosakot | Daviey: wow, what is this animation?!? :) | 20:24 |
| Daviey | it's coming back | 20:25 |
| vhosakot | yeah I see it | 20:25 |
| *** huikang has quit IRC | 20:26 | |
| Daviey | vhosakot: ok | 20:28 |
| Daviey | vhosakot: it worked | 20:28 |
| Daviey | vhosakot: are you ok to take over now? | 20:28 |
| Daviey | vhosakot: next step is to create a new inventory file, and try and get docker setup and installed on this node | 20:29 |
| Daviey | copy over the ssh key etc | 20:29 |
| vhosakot | Daviey: great! thanks a lot for your help... | 20:29 |
| vhosakot | we can use the same inventory file we used for ubuntu | 20:29 |
| Daviey | vhosakot: Once we ar ehappy that docker is working on this single node, we can reproduce the whole thing across the entire cluster | 20:29 |
| *** Satya has joined #openstack-kolla | 20:29 | |
| Satya | Hi all | 20:29 |
| Daviey | vhosakot: yeah, but start with JUST this node | 20:30 |
| Daviey | docker needs to be setup slightly differently etc | 20:30 |
| vhosakot | Daviey: should I install kolla on this node and test? or just install docker and test ? | 20:30 |
| Satya | facing an issue where nova resize failed as nova is not a part of sudoer | 20:30 |
| Daviey | just docker | 20:30 |
| Satya | can anyone help? | 20:30 |
| Daviey | Satya: try rebuilding your images... we've seen this happen a few times now | 20:31 |
| vhosakot | Daviey: I know the steps to install docker (it is the kolla quick start guide).. is it enough if I do just that ? | 20:31 |
| Daviey | vhosakot: kolla can stick on this the same control node we used for ubuntu | 20:31 |
| vhosakot | Daviey: you mean the deploy node ? | 20:31 |
| Daviey | vhosakot: Althought it would be nicer to prove it works with centos as a control node, i don't think that is part of the test | 20:31 |
| Daviey | vhosakot: yeah | 20:32 |
| vhosakot | Daviey: right, I think CentOS on target nodes is a good test of OpenSTack deployed by kolla on centos | 20:32 |
| Satya | 'sudo', 'privsep-helper', '--config-file', '/etc/nova/nova.conf', '--privsep_context', 'os_brick.privileged.default', '--privsep_sock_path', '/tmp/tmpTeH3JX/privsep.sock' | 20:32 |
| Satya | this is the command which run on the compute node | 20:32 |
| Satya | which exited as non zero | 20:33 |
| vhosakot | who is in tmux | 20:33 |
| Satya | added nova to sudoer and it works | 20:33 |
| Satya | but is there a fix in kolla | 20:33 |
| Satya | any help? | 20:34 |
| Satya | britt u there? | 20:35 |
| *** Jeffrey4l_ has quit IRC | 20:35 | |
| Daviey | vhosakot: master has a playbook for setting up the nodes | 20:36 |
| *** Jeffrey4l_ has joined #openstack-kolla | 20:36 | |
| Satya | britthou_ | 20:36 |
| *** xwang has joined #openstack-kolla | 20:37 | |
| *** britthou_ is now known as britthouser | 20:37 | |
| *** jtriley_ has quit IRC | 20:38 | |
| *** xwang_ has joined #openstack-kolla | 20:39 | |
| *** xwang has quit IRC | 20:39 | |
| vhosakot | yep | 20:39 |
| *** Satya has quit IRC | 20:41 | |
| xwang_ | Hello, can anyone help me with kolla deployment? I got stuck for a while | 20:42 |
| xwang_ | I'm trying to run # kolla-ansible deploy | 20:43 |
| *** daneyon has joined #openstack-kolla | 20:43 | |
| xwang_ | But at the step "TASK [neutron : Waiting the openvswitch_db service to be ready]" | 20:43 |
| xwang_ | It kept failing | 20:45 |
| xwang_ | TASK [neutron : Waiting the openvswitch_db service to be ready] **************** FAILED - RETRYING: TASK: neutron : Waiting the openvswitch_db service to be ready (29 retries left). FAILED - RETRYING: TASK: neutron : Waiting the openvswitch_db service to be ready (28 retries left). FAILED - RETRYING: TASK: neutron : Waiting the openvswitch_db service to be ready (27 retries left). FAILED - RETRYING: TASK: neutron : Waiting th | 20:45 |
| *** senk has quit IRC | 20:46 | |
| xwang_ | The error message is fatal: [localhost]: FAILED! => {"changed": false, "cmd": ["docker", "exec", "openvswitch_db", "ovs-vsctl", "--no-wait", "show"], "delta": "0:00:00.013437", "end": "2016-08-29 12:54:46.933598", "failed": true, "rc": 1, "start": "2016-08-29 12:54:46.920161", "stderr": "Error response from daemon: Container 37e16719a963ae2561db6df932ee274f55164577d6db4c5e73cbdf8ee469381b is restarting, wait until the contain | 20:46 |
| *** daneyon has quit IRC | 20:48 | |
| Daviey | vhosakot: whatcha trying to do? | 20:48 |
| vhosakot | Daviey: sorry, I was trying to see which servers got bootstrapped | 20:48 |
| xwang_ | I was trying "kolla-ansible deploy" on a virtual machine | 20:49 |
| Daviey | vhosakot: just the one | 20:49 |
| Daviey | vhosakot: the one we are working on at the moment | 20:49 |
| vhosakot | Daviey: yep.. cool | 20:50 |
| Daviey | vhosakot: the one we just installed | 20:50 |
| Daviey | vhosakot: ok, we look good now | 20:51 |
| *** salv-orlando has joined #openstack-kolla | 20:51 | |
| Daviey | vhosakot: are you ok to reinstall all the machines doing what we just did? | 20:52 |
| vhosakot | Daviey: is this now installing all-in-one kolla on 0.23 ? | 20:52 |
| *** sdake has joined #openstack-kolla | 20:52 | |
| Daviey | vhosakot: no! | 20:53 |
| vhosakot | then ? | 20:53 |
| Daviey | vhosakot: I just kicked off an image build for centos | 20:53 |
| Daviey | and bugger, i didn't push them to the registrt | 20:53 |
| vhosakot | Daviey: on the deploy node | 20:53 |
| vhosakot | Daviey: what about installing CentOS on the 130 nodes... ? I'll get started with that | 20:54 |
| Daviey | that is what i mean | 20:54 |
| Daviey | the cobbler stuff | 20:54 |
| Daviey | we've proved it works for 1, so i'm happy it will work for all | 20:54 |
| vhosakot | Daviey: well, I didn;t watch the commands you ran.. I'll need your help or can you point me to the commands in history... | 20:55 |
| vhosakot | you already did cobbler sync right ? | 20:55 |
| Daviey | vhosakot: cobbler sync will need to be run before deploy | 20:56 |
| vhosakot | Daviey: cool.. | 20:56 |
| Daviey | vhosakot: inc0 removed all the nodes earlier, not sure why | 20:56 |
| Daviey | but all the systems will need to be created in cobbler | 20:56 |
| vhosakot | Daviey: "cobbler system list" shows just "729583-comp-s3700-023.cloud7.osic.rackspace.com".. is there a script to add the 130 cobbler systems? | 20:57 |
| *** salv-orlando has quit IRC | 20:59 | |
| vhosakot | Daviey: ^^ | 20:59 |
| Daviey | vhosakot: yeah... but i don't have it | 21:00 |
| Daviey | would be really helpful if inc0 woke back up | 21:00 |
| vhosakot | right | 21:01 |
| vhosakot | inc0: is there a script to add the 130 systems in cobbler | 21:01 |
| vhosakot | Daviey: what IP did you enter in the kolla inventory file that is currently building ? | 21:01 |
| vhosakot | we can use the same inventory file (for 130 nodes) we used for ubuntu ? | 21:02 |
| vhosakot | who is in tmux now ? | 21:03 |
| Daviey | me | 21:03 |
| Daviey | vhosakot: we can yes, but i wanted an inventory file with just one host in to test instaling docker | 21:04 |
| vhosakot | cool | 21:04 |
| *** b_bezak has quit IRC | 21:09 | |
| Daviey | vhosakot: are you following? | 21:11 |
| *** huikang_ has quit IRC | 21:11 | |
| *** huikang has joined #openstack-kolla | 21:12 | |
| *** huikang_ has joined #openstack-kolla | 21:13 | |
| vhosakot | Daviey: yes, I see 129 systems now... generated form the CSV.. cool | 21:14 |
| sdake | thanks a slew Daviey vhosakot | 21:14 |
| sdake | sounds like you are both close :) | 21:14 |
| *** inc0 has quit IRC | 21:15 | |
| sdake | the hard pat will be networking config on each node | 21:15 |
| Daviey | sdake: nah | 21:15 |
| openstackgerrit | Kevin Fox proposed openstack/kolla-kubernetes: Remove unneeded config overrides. https://review.openstack.org/362449 | 21:15 |
| *** salv-orlando has joined #openstack-kolla | 21:16 | |
| *** huikang has quit IRC | 21:16 | |
| vhosakot | Daviey: cool, the boot devices are now set... | 21:17 |
| *** huikang_ has quit IRC | 21:17 | |
| Daviey | vhosakot: hopefully | 21:18 |
| Daviey | vhosakot: so we are reinstalling the one we already installed aswell | 21:18 |
| vhosakot | Daviey: cool... | 21:20 |
| Daviey | bugger didn't work | 21:20 |
| *** kangh__ has quit IRC | 21:24 | |
| *** huikang has joined #openstack-kolla | 21:25 | |
| openstackgerrit | Kevin Fox proposed openstack/kolla-kubernetes: Make libvirt over tcp an option https://review.openstack.org/362457 | 21:26 |
| vhosakot | Daviey: the tftpboot config for all the 130 nodes are same except the MAC address and the IP address, right ? | 21:26 |
| Daviey | vhosakot: yes | 21:27 |
| vhosakot | cool | 21:27 |
| *** huikang has quit IRC | 21:30 | |
| *** sdake has quit IRC | 21:33 | |
| vhosakot | Daviey: I see they are powercycling.. cool | 21:39 |
| Daviey | vhosakot: yeah, i think it is workin | 21:40 |
| vhosakot | Daviey: once done, can we login into few nods and check.. ? | 21:42 |
| vhosakot | nodes* | 21:42 |
| Daviey | vhosakot: sure | 21:42 |
| Daviey | vhosakot: infact, copying the ssh key to all nodes is a good test :) | 21:42 |
| vhosakot | Daviey: yep, I think there is a playbook fo that | 21:42 |
| Daviey | vhosakot: ansible -k -i ./inventory -m authorized_key -a "key=\"{{lookup('file','~/.ssh/id_rsa.pub')}}\" user=$USER" all | 21:43 |
| *** aNupoisc has quit IRC | 21:44 | |
| vhosakot | cool | 21:44 |
| *** aNupoisc has joined #openstack-kolla | 21:44 | |
| *** xwang_ has quit IRC | 21:49 | |
| vhosakot | Daviey: can we try pinging 151 ? | 21:51 |
| Daviey | sure | 21:51 |
| Daviey | vhosakot: err that works | 21:51 |
| Daviey | i just did | 21:51 |
| vhosakot | Daviey: may sshd is down on the targets | 21:51 |
| Daviey | vhosakot: i suspect it is still instaling | 21:51 |
| openstackgerrit | Serguei Bezverkhi proposed openstack/kolla-kubernetes: Script generating kubernetes secrets from passwords.yml https://review.openstack.org/354199 | 21:52 |
| vhosakot | Daviey: yep.. CentOS baremetal boot takes few mins... | 21:52 |
| openstackgerrit | Merged openstack/kolla-kubernetes: Deployment, Readiness, Safe Shutdown, & Scaling for glance https://review.openstack.org/354895 | 21:52 |
| openstackgerrit | Serguei Bezverkhi proposed openstack/kolla-kubernetes: Script generating kubernetes secrets from passwords.yml https://review.openstack.org/354199 | 21:55 |
| Daviey | vhosakot: there we go | 21:57 |
| vhosakot | :) | 21:57 |
| *** banix has quit IRC | 21:58 | |
| openstackgerrit | Ryan Hallisey proposed openstack/kolla-kubernetes: Update the quickstart https://review.openstack.org/362325 | 21:59 |
| openstackgerrit | Ryan Hallisey proposed openstack/kolla-kubernetes: Add a script that will get the latest kubectl https://review.openstack.org/362312 | 21:59 |
| openstackgerrit | Ryan Hallisey proposed openstack/kolla-kubernetes: Update the kubernetes aio guide https://review.openstack.org/361787 | 21:59 |
| openstackgerrit | Ryan Hallisey proposed openstack/kolla-kubernetes: Massive rework of the kolla-kubernetes docs https://review.openstack.org/362467 | 21:59 |
| *** xwang has joined #openstack-kolla | 22:00 | |
| vhosakot | Daviey: !!! | 22:01 |
| Daviey | vhosakot: ?? | 22:01 |
| vhosakot | I see it worked on all 151 nodes! | 22:01 |
| vhosakot | 129 I mean | 22:01 |
| Daviey | vhosakot: yeah, looks good | 22:01 |
| *** rhallisey has quit IRC | 22:01 | |
| Daviey | vhosakot: Are you ok to take over now? | 22:01 |
| vhosakot | yes, now, once the images are built on the deploy node (that is currently going on), I'll install kolla on all the nodes.. | 22:02 |
| Daviey | vhosakot: the images might need to be rebuilt and pushed to the registry | 22:02 |
| *** gfhellma1 has joined #openstack-kolla | 22:03 | |
| vhosakot | Daviey: yep, I can do that | 22:03 |
| Daviey | i fear the registry might not have enough space | 22:03 |
| Daviey | vhosakot: do you know the command to push? | 22:03 |
| *** gfhellma has quit IRC | 22:03 | |
| vhosakot | Daviey: "729494-comp-s3500-002" is the deploy node right.. I 've opened it in the tmux session | 22:03 |
| vhosakot | Daviey: cool I'll use --push | 22:03 |
| vhosakot | Daviey: can we check the space of /var/lib/docker to make sire deploy node has enough space | 22:04 |
| vhosakot | coo, I see you already started the push | 22:04 |
| Daviey | vhosakot: sorry, you really do have the driving seat now | 22:04 |
| vhosakot | Daviey: yep, I'll take it from here.. thanks a lot Daviey! | 22:05 |
| Daviey | vhosakot: cool | 22:05 |
| openstackgerrit | Serguei Bezverkhi proposed openstack/kolla-kubernetes: Script generating kubernetes secrets from passwords.yml https://review.openstack.org/354199 | 22:05 |
| vhosakot | Daviey: no space :( | 22:05 |
| vhosakot | # df -h /var/lib/docker | 22:05 |
| vhosakot | Filesystem Size Used Avail Use% Mounted on | 22:05 |
| vhosakot | /dev/mapper/lxc-root00 47G 45G 0 100% / | 22:05 |
| Daviey | vhosakot: delete the ubuntu images | 22:06 |
| vhosakot | Daviey: r u sure.. we dont need them right ? | 22:06 |
| Daviey | vhosakot: we need to give the cluster back tomorrow, so unlikely | 22:06 |
| Daviey | and if we do, they build in 20 mins or so | 22:06 |
| vhosakot | Daviey: cool.. thanks! | 22:06 |
| vhosakot | right | 22:06 |
| *** ayoung has quit IRC | 22:07 | |
| Daviey | hmm | 22:07 |
| *** Pavo has quit IRC | 22:07 | |
| vhosakot | "rmi -f" will delete forcefully | 22:08 |
| vhosakot | cool! | 22:09 |
| vhosakot | delete the volumes and the stop the running containers as well | 22:10 |
| vhosakot | and delete everything in "docker ps -a " | 22:10 |
| vhosakot | Daviey: why is 75% still used ? | 22:14 |
| vhosakot | Daviey: let us restart docker and try | 22:14 |
| Daviey | yep | 22:15 |
| vhosakot | restarting docker frees up some space sometimes | 22:15 |
| vhosakot | ah! | 22:15 |
| Daviey | vhosakot: lets see if we have enoug already | 22:16 |
| vhosakot | Daviey: cool | 22:16 |
| vhosakot | 25% of 47G should be good I think | 22:17 |
| vhosakot | Daviey: the build output looks good! | 22:17 |
| vhosakot | I will check later tonight.. thanks a lot for your help! | 22:17 |
| Daviey | vhosakot: i might be in bed then | 22:18 |
| Daviey | vhosakot: let me know how you get on | 22:18 |
| *** chrichip has joined #openstack-kolla | 22:18 | |
| vhosakot | Daviey: no.. we'll chat tomorrow.. thanks | 22:18 |
| *** Pavo has joined #openstack-kolla | 22:18 | |
| Daviey | vhosakot: keep in mind we are supposed to hand them back tomorrow | 22:18 |
| *** zhiwei has joined #openstack-kolla | 22:18 | |
| vhosakot | Daviey: what time ? | 22:18 |
| Daviey | vhosakot: so as much as can get done in the next few hours is worth it | 22:18 |
| vhosakot | Daviey: we can run one baseline test tonight | 22:18 |
| Daviey | vhosakot: during the business day :/ | 22:19 |
| *** gfhellma has joined #openstack-kolla | 22:19 | |
| *** gfhellma1 has quit IRC | 22:20 | |
| *** dwalsh has joined #openstack-kolla | 22:22 | |
| *** michauds has quit IRC | 22:23 | |
| *** zhiwei has quit IRC | 22:23 | |
| openstackgerrit | Kevin Fox proposed openstack/kolla-kubernetes: Remove unneeded config overrides. https://review.openstack.org/362449 | 22:24 |
| *** chrichip has quit IRC | 22:25 | |
| *** signed8b_ has joined #openstack-kolla | 22:28 | |
| *** signed8bit has quit IRC | 22:30 | |
| *** huikang has joined #openstack-kolla | 22:33 | |
| *** fragatin_ has joined #openstack-kolla | 22:35 | |
| Pavo | evening everyone | 22:36 |
| *** huikang has quit IRC | 22:38 | |
| *** fragatina has quit IRC | 22:38 | |
| *** huikang has joined #openstack-kolla | 22:39 | |
| *** bmace has quit IRC | 22:40 | |
| *** huikang has quit IRC | 22:43 | |
| xwang | Hello | 22:54 |
| xwang | Evening Pavo | 22:54 |
| *** huikang has joined #openstack-kolla | 22:59 | |
| *** mark-casey has quit IRC | 23:03 | |
| *** ntpttr has quit IRC | 23:08 | |
| *** ntpttr has joined #openstack-kolla | 23:08 | |
| *** huikang has quit IRC | 23:11 | |
| *** huikang has joined #openstack-kolla | 23:12 | |
| *** vhosakot has quit IRC | 23:14 | |
| *** huikang has quit IRC | 23:16 | |
| *** ayoung has joined #openstack-kolla | 23:20 | |
| sbezverk_ | vhosakot: ping | 23:23 |
| *** rkrum has joined #openstack-kolla | 23:24 | |
| *** salv-orlando has quit IRC | 23:25 | |
| *** daneyon has joined #openstack-kolla | 23:26 | |
| *** dwalsh has quit IRC | 23:29 | |
| *** daneyon has quit IRC | 23:30 | |
| openstackgerrit | Merged openstack/kolla-kubernetes: Update the kubernetes aio guide https://review.openstack.org/361787 | 23:35 |
| openstackgerrit | Merged openstack/kolla-kubernetes: Add a script that will get the latest kubectl https://review.openstack.org/362312 | 23:40 |
| openstackgerrit | Merged openstack/kolla-kubernetes: Update the quickstart https://review.openstack.org/362325 | 23:40 |
| *** huikang has joined #openstack-kolla | 23:41 | |
| *** zhurong has joined #openstack-kolla | 23:44 | |
| *** huikang has quit IRC | 23:44 | |
| *** huikang has joined #openstack-kolla | 23:45 | |
| *** srwilkers_ has joined #openstack-kolla | 23:48 | |
| *** srwilkers_ has left #openstack-kolla | 23:48 | |
| *** huikang has quit IRC | 23:49 | |
| *** srwilkers__ has quit IRC | 23:50 | |
| *** huikang has joined #openstack-kolla | 23:51 | |
| *** frzn_ has joined #openstack-kolla | 23:51 | |
| *** huikang has quit IRC | 23:57 | |
Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!
