Saturday, 2018-04-28

*** spine55 has quit IRC00:02
*** dxiri has quit IRC00:07
*** admin0 has quit IRC00:30
*** admin0 has joined #openstack-ansible00:31
openstackgerritKevin Carter (cloudnull) proposed openstack/openstack-ansible-ops master: [WIP] Add XPACK monitoring to ELK and Beats  https://review.openstack.org/56492600:36
*** admin0 has quit IRC00:38
*** admin0 has joined #openstack-ansible00:39
*** lbragstad has joined #openstack-ansible00:40
openstackgerritKevin Carter (cloudnull) proposed openstack/openstack-ansible-ops master: Add options for better memory management  https://review.openstack.org/56495500:44
*** admin0 has quit IRC00:44
*** admin0 has joined #openstack-ansible00:45
*** armaan has quit IRC00:51
*** lbragstad has quit IRC00:51
*** armaan has joined #openstack-ansible00:52
openstackgerritKevin Carter (cloudnull) proposed openstack/openstack-ansible master: Convert rsyslog to an include_task with group_vars  https://review.openstack.org/55600200:53
openstackgerritKevin Carter (cloudnull) proposed openstack/openstack-ansible-ops master: Add options for better memory management  https://review.openstack.org/56495501:13
*** admin0 has quit IRC01:22
*** cmart has quit IRC01:29
*** mma has joined #openstack-ansible01:45
*** tomtom001 has quit IRC01:46
*** tomtom001 has joined #openstack-ansible01:46
*** mma has quit IRC01:49
*** yifei has joined #openstack-ansible02:06
*** dave-mccowan has joined #openstack-ansible02:09
*** dave-mccowan has quit IRC02:35
*** tomtom001 has quit IRC02:59
*** qwebirc95616 has joined #openstack-ansible02:59
*** tomtom001 has joined #openstack-ansible03:05
*** yifei has quit IRC03:45
*** jwitko has joined #openstack-ansible03:51
openstackgerritKevin Carter (cloudnull) proposed openstack/openstack-ansible-ops master: Add options for better memory management  https://review.openstack.org/56495503:58
*** gkadam has joined #openstack-ansible04:16
openstackgerritKevin Carter (cloudnull) proposed openstack/openstack-ansible-ops master: Add options for better memory management  https://review.openstack.org/56495504:29
*** mma has joined #openstack-ansible04:44
*** mma has quit IRC04:49
*** tomtom001 has quit IRC05:00
*** tomtom001 has joined #openstack-ansible05:01
*** gyee has quit IRC05:25
*** yolanda has quit IRC05:41
*** yolanda has joined #openstack-ansible05:42
*** harryhuang_ has quit IRC06:59
*** spsurya has joined #openstack-ansible07:30
*** armaan has quit IRC08:23
*** armaan has joined #openstack-ansible08:24
*** armaan has quit IRC08:45
*** armaan has joined #openstack-ansible08:45
evrardjpI am kinda stuck with no machine big enough to test patches, I didn't realize it would be so painful :p08:46
*** armaan has quit IRC09:03
*** armaan has joined #openstack-ansible09:04
*** l4yerffej has quit IRC09:34
*** l4yerffej has joined #openstack-ansible09:35
*** l4yerffej has quit IRC09:41
*** l4yerffej has joined #openstack-ansible09:43
*** l4yerffej has quit IRC09:48
*** l4yerffej has joined #openstack-ansible09:51
*** tomtom001 has quit IRC10:00
*** tomtom001 has joined #openstack-ansible10:01
*** l4yerffej has quit IRC10:10
*** l4yerffej has joined #openstack-ansible10:12
*** masuberu has joined #openstack-ansible10:19
*** masber has quit IRC10:23
*** l4yerffej has quit IRC10:30
*** l4yerffej has joined #openstack-ansible10:47
*** nicolasbock has joined #openstack-ansible11:06
*** armaan has quit IRC11:06
*** armaan has joined #openstack-ansible11:14
*** tosky has joined #openstack-ansible11:20
*** jwitko has quit IRC11:47
*** evin has quit IRC11:58
*** armaan has quit IRC12:19
*** armaan has joined #openstack-ansible12:20
openstackgerritMerged openstack/openstack-ansible-os_tempest master: Comment tempest_git_repo role default  https://review.openstack.org/56476412:22
openstackgerritMatt Thompson proposed openstack/openstack-ansible-os_tempest stable/queens: Comment tempest_git_repo role default  https://review.openstack.org/56501812:26
openstackgerritMatt Thompson proposed openstack/openstack-ansible-os_tempest stable/pike: Comment tempest_git_repo role default  https://review.openstack.org/56501912:27
*** spsurya has quit IRC12:44
*** eumel8 has quit IRC12:58
nicolasbockI found that cache-prep-commands.sh gets stuck on leap 42.313:03
nicolasbockThe issue is that the zypper command in the opensuse chroot gets stuck in the refresh phase for some reason13:03
nicolasbockWhen I `mout --rbind /dev /var/lib/machines/opensuse-42.3-amd64/dev` it works13:03
nicolasbockhwoarang, what do you think? ^^13:05
nicolasbockI run the mount command as the zypper command in the chroot is stuck13:19
nicolasbockThis unstucks it13:19
*** armaan has quit IRC13:28
*** armaan has joined #openstack-ansible13:29
*** armaan has quit IRC13:33
*** mgariepy has quit IRC13:50
*** esberglu has joined #openstack-ansible14:21
*** esberglu has quit IRC14:22
*** vnogin has joined #openstack-ansible14:36
*** armaan has joined #openstack-ansible14:41
openstackgerritJean-Philippe Evrard proposed openstack/openstack-ansible master: [DNM] Test integrated build  https://review.openstack.org/56502514:46
evrardjphey nicolasbock14:46
evrardjpsorry for the late answer14:46
evrardjpthe lxc cache prep commands are implemented per distro, and we've seen it stuck occassionally in the past, but it seems we haven't got issues recently.14:47
evrardjplet me double check in periodic gates real quick14:47
evrardjpnicolasbock: do you know why you'd need the dev mountpoint ?14:48
*** vnogin has quit IRC15:11
openstackgerritJean-Philippe Evrard proposed openstack/openstack-ansible master: Freeze roles for Milestone 1  https://review.openstack.org/56502715:17
*** tomtom001 has quit IRC15:17
*** tomtom001 has joined #openstack-ansible15:18
*** masber has joined #openstack-ansible15:20
*** masuberu has quit IRC15:23
nicolasbockHi evrardjp15:24
nicolasbockNo unfortunately I don't15:24
nicolasbockI can try to find out why zypper might need something in /dev15:24
nicolasbockI see that it goes into updating the repo and the [.... progress thingy gets stuck at some point eary on15:25
nicolasbockThen I mount /dev into the chroot and it continues15:25
*** armaan has quit IRC15:27
*** armaan has joined #openstack-ansible15:28
evrardjpI haven't seen that in gates though15:28
evrardjpdo you have a specific override?15:28
*** vnogin has joined #openstack-ansible15:29
openstackgerritJean-Philippe Evrard proposed openstack/openstack-ansible stable/queens: Update all SHAs for 17.0.4  https://review.openstack.org/56502915:29
nicolasbockI am running inside a kvm VM locally.15:33
nicolasbockThe OS is a basic Leap 42.315:33
evrardjpnicolasbock: maybe worth checking if there is a difference between gate check commit and your proces15:48
evrardjps15:48
nicolasbockTrue15:48
evrardjpscripts/gate-check-commit.sh aio_lxc deploy15:48
nicolasbockThanks15:49
evrardjpassuming you're deploying a standard aio with lxc :p15:49
nicolasbockYes, I am deploying lxc_metal15:49
evrardjpif you don't want to run this you can simply check with our latest logs15:49
evrardjpaio_metal I guess :)15:49
nicolasbockTrue :)15:50
evrardjpnicolasbock: let me add a aio_metal for suse15:50
nicolasbockoh nice15:50
nicolasbockThanks!15:50
evrardjpI will do that when i have finished releases :)15:50
nicolasbockSure, no rush :)15:50
evrardjpnicolasbock: please note that if you're doing full aio_metal, there is no reason you should enter that code15:52
evrardjpIIRC15:52
evrardjpnicolasbock: you are talking about the lxc|nspawn cache prep?15:52
*** vnogin has quit IRC15:54
nicolasbockHmm, I guess I am not using aio_metal. I thought that I had set SCENARIO=aio_metal in my env but checking now it seems that I haven't15:56
openstackgerritJean-Philippe Evrard proposed openstack/openstack-ansible stable/pike: Update all SHAs for 16.0.13  https://review.openstack.org/56503416:01
evrardjpnicolasbock: in any case, both issues should be solved.16:02
evrardjpboth means for me 1) the fact we don't have aio_metal tested in suse 2) your issue with aio that doesn't seem to happen in gates16:02
*** vnogin has joined #openstack-ansible16:03
openstackgerritKevin Carter (cloudnull) proposed openstack/openstack-ansible-ops master: Add beats-setup tag  https://review.openstack.org/56503516:04
*** gkadam has quit IRC16:08
*** vnogin has quit IRC16:08
*** vnogin has joined #openstack-ansible16:11
*** vnogin has quit IRC16:12
evrardjpcloudnull: https://review.openstack.org/#/c/564070/ :)16:17
evrardjpcloudnull: if you need anything reviewed, ping me :)16:18
cloudnullill trade you https://review.openstack.org/#/c/564530/ :)16:18
cloudnullevrardjp: is grey log not compatible with elastic 6.x ?16:22
cloudnullhttps://review.openstack.org/#/c/564070/2/graylog/graylog2-install.yml16:22
openstackgerritJean-Philippe Evrard proposed openstack/openstack-ansible stable/ocata: Update all SHAs for 15.1.21  https://review.openstack.org/56503716:23
evrardjpcloudnull: nope :/16:23
cloudnullbummer.16:23
evrardjpnot yet16:23
evrardjpbut that would be super nice16:24
evrardjpI will wait for the other roles upstream to move to elastic 616:24
evrardjpthen we can converge to the same thing16:24
evrardjpI've sadly discovered that metrics for memcached aren't great.16:25
cloudnullthe stuff we have in the ops repo is quite nice :)16:25
evrardjpAnd I heard people are integrating OSA with "mirantis" monitoring tooling, which is apparently very efficient. ppetit worked on it :)16:26
evrardjpif it's complete an efficient and upstreamable, that could also go there as another way to monitor things16:26
cloudnull++16:27
evrardjpso we would have in user stories "integrating x with OSA"16:27
evrardjpwhere x is whatever you pick: monitoring with monitorstack+TIGCK stack, zabbix, whatever...16:27
evrardjpmonitoring|metrics|whatever topic16:28
openstackgerritMerged openstack/openstack-ansible-ops master: Add Graylog Central Logging  https://review.openstack.org/56407016:28
cloudnull++ i think that would be great16:28
cloudnullin the land of Rackspace Private Cloud we're about to drop our custom elk roles and pivot to whats in the ops repo.16:29
evrardjpsounds good. Just pivot to what's there, and contribute to keep it up to date16:30
cloudnull++16:30
evrardjpeventually include test upstream, else if possible include tests downstream with external CI16:30
evrardjpbut there is no urgency for ops tests, the most stable thing should be the integrated repo16:30
cloudnullyea. IDK what the best thing would be for ops repo tests16:31
cloudnullmaybe we could do something like we have for docs where specific tests are run when specific directories are modified ?16:31
evrardjpcloudnull: yeah that's what I meant16:33
* cloudnull - aside - is loving beats - https://snag.gy/eiCqb6.jpg, https://snag.gy/yD5Yfc.jpg16:33
cloudnullI've got all of this that running on 20+ node environment (random old servers) and it's lovely16:34
evrardjpthat looks nice, what's the beats configuration side?16:34
cloudnullthat's packet beat16:35
evrardjpI mean it looks like you're instrumenting things, but I don't know how16:35
evrardjpoh ok16:35
evrardjpso that's with libpcap right?16:35
evrardjpIIRC what you told me in the past16:35
cloudnullI have it set to af_packet, but yes.16:35
*** mgariepy has joined #openstack-ansible16:36
evrardjpthat's scary though :)16:37
evrardjpwell I guess as long as we don't have mysql + ssl everything will be transparent16:37
cloudnullyup.16:37
evrardjpsame for rabbit, memcache, and others16:37
cloudnullrabbit is all ssl so i dont see the messages, but i do see the rates16:38
evrardjpbut it's less intrusive than changing at the app level16:38
evrardjpok16:38
cloudnullalso https://snag.gy/bBolDq.jpg16:38
evrardjpthat's proper graphing16:38
evrardjphow is the ES cluster?16:38
evrardjpNot too much data?16:38
cloudnullwhich is using grafana from the ops repo w/ our elk things16:38
evrardjpI guess you want to use it for live data only?16:38
evrardjpOr want to use it for trends later ?16:39
evrardjpor past search16:39
cloudnullall of the above16:39
evrardjphaha16:39
evrardjpthen yeah, hard for the ES cluster in the long term :)16:39
evrardjpit's gonna be interesting!16:39
cloudnullthe grafana dashboards were uploaded to https://grafana.com/dashboards so installing them and getting it all live is super simple16:40
evrardjpthat's great! That's what I wanted to do during the leapfrogs16:40
evrardjpcloudnull: I can't seem to find them by searching openstack16:42
evrardjpthat's somethign you might want to improve :)16:42
evrardjpSEO all the things!16:42
evrardjp:D16:42
*** bhujay has joined #openstack-ansible16:43
qwebirc95616hey guys! i have deployed openstack-ansible but i have issue with launching instance on second compute node. i use stable/pike and i have two compute node with one controller and one node as storage. the exact error is it return : "  TypeError: 'NoneType' object has no attribute '__getitem__' " could anyone help me plzzz16:43
evrardjpqwebirc95616: that's weird, what are you using to launch the instance16:44
evrardjpqwebirc95616: and what's the SHA on your stable/pike branch?16:44
evrardjpI have to go, else my wife won't be happy of me working the whole afternoon :p16:45
qwebirc95616evrardjp i've tried from terminal to launch instance.16:45
evrardjpqwebirc95616: from the utility container?16:45
evrardjpusing our virtualenv ?16:46
qwebirc95616evrardjp yes16:46
evrardjpqwebirc95616: I suggest you fill all of this in a bug: telling how you deployed (which sha, which overrides), and how you started this instance (showing the shell history for example, to help reproduce the bug)16:47
evrardjpand a full log of the issue16:47
qwebirc95616evrardjp thanks :)16:47
evrardjpqwebirc95616: it seems not normal, as Pike deployed recently just fine, so either you have an old release, or you have something unclean on the node16:47
evrardjplike a conflicting version of python/pip16:47
evrardjpor not using the venv16:48
evrardjpall of that is possible, so that's why logs matter :)16:48
evrardjpa full trace of the log is good too.16:48
evrardjpdon't hesitate to put the logs on paste.openstack.org16:48
evrardjpthat's there for that :)16:48
evrardjpsorry I can't help you more right now, and with those details. But we'll get through it :)16:48
evrardjpsee you all laterz!16:49
qwebirc95616evrardjp thank you so much i will do your suggestion16:49
qwebirc95616here is the full log http://paste.openstack.org/show/720071/16:51
cloudnullevrardjp: it doesn't look like I can add an OpenStack tag to the dashboards. I added a blerb in the description though17:07
cloudnullok, i gotta run17:07
* cloudnull off to be bob the builder.17:07
cloudnullif anything need be reviewed let me know, I'll be back online later.17:08
*** bhujay has quit IRC17:23
*** qwebirc95616 has quit IRC17:42
*** bhujay has joined #openstack-ansible17:48
*** nicolasbock has quit IRC17:51
*** vnogin has joined #openstack-ansible17:57
evrardjp[m]the milestone 1 patch if possible, i will ping that. thanks for the offer Bob ! ;)18:04
*** vnogin has quit IRC18:11
*** vnogin has joined #openstack-ansible18:16
*** vnogin has quit IRC18:16
*** mma has joined #openstack-ansible18:23
*** bhujay has quit IRC18:44
openstackgerritJean-Philippe Evrard proposed openstack/openstack-ansible-lxc_hosts master: Avoid release note conflict  https://review.openstack.org/56504518:52
evrardjpcloudnull: voting on https://review.openstack.org/#/c/565045/ and https://review.openstack.org/#/c/565027/1 deserve votes18:54
*** armaan has quit IRC18:56
*** armaan has joined #openstack-ansible18:56
*** armaan has quit IRC19:02
*** armaan has joined #openstack-ansible19:02
*** jbadiapa has quit IRC19:18
*** armaan has quit IRC19:20
*** armaan has joined #openstack-ansible19:20
*** vnogin has joined #openstack-ansible20:01
*** vnogin has quit IRC20:11
*** armaan has quit IRC20:42
*** armaan has joined #openstack-ansible20:58
*** DanyC has joined #openstack-ansible21:23
*** throwsb1 has joined #openstack-ansible21:27
*** armaan has quit IRC21:32
*** armaan has joined #openstack-ansible21:40
*** vnogin has joined #openstack-ansible21:46
*** DanyC has quit IRC21:51
*** jbadiapa has joined #openstack-ansible21:55
*** mma has quit IRC21:56
*** mwarad has quit IRC22:06
*** evin has joined #openstack-ansible22:13
*** vnogin has quit IRC22:20
*** jbadiapa has quit IRC23:18
*** tosky has quit IRC23:26
*** throwsb1 has quit IRC23:36
*** evin has quit IRC23:54

Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!