Monday, 2017-03-20

*** dave-mccowan has joined #openstack-infra00:14
*** dingyichen has joined #openstack-infra00:15
*** kzaitsev_mb has quit IRC00:18
*** baoli has quit IRC00:18
*** baoli has joined #openstack-infra00:22
*** claudiub has quit IRC00:30
*** aeng has quit IRC00:33
*** jamielennox is now known as jamielennox|away00:38
*** yamamoto has joined #openstack-infra00:41
*** yamamoto has quit IRC00:45
*** aeng has joined #openstack-infra00:51
*** dingyichen has quit IRC00:55
*** dingyichen has joined #openstack-infra00:56
*** tosky has quit IRC00:58
*** Julien-zte has joined #openstack-infra01:01
*** baoli has quit IRC01:04
*** mriedem has quit IRC01:08
*** yamamoto has joined #openstack-infra01:11
*** dingyichen has quit IRC01:14
*** dingyichen has joined #openstack-infra01:14
*** kzaitsev_mb has joined #openstack-infra01:15
*** yamamoto has quit IRC01:15
*** gongysh has joined #openstack-infra01:16
*** aeng has quit IRC01:17
*** rfolco has joined #openstack-infra01:23
*** rfolco has quit IRC01:23
*** liujiong has joined #openstack-infra01:26
*** yamamoto has joined #openstack-infra01:28
*** yamamoto has quit IRC01:28
*** baoli has joined #openstack-infra01:30
*** aeng has joined #openstack-infra01:30
*** Julien-zte has quit IRC01:34
*** EricGonczer_ has quit IRC01:34
*** yamamoto has joined #openstack-infra01:36
*** kaisers1 has quit IRC01:36
*** kaisers2 has joined #openstack-infra01:36
*** rfolco has joined #openstack-infra01:39
*** rfolco has quit IRC01:39
*** yamamoto has quit IRC01:41
*** pahuang has joined #openstack-infra01:45
*** markvoelker has joined #openstack-infra01:45
*** jiaohaolin1 has joined #openstack-infra01:46
*** markvoelker has quit IRC01:49
*** rfolco has joined #openstack-infra01:54
*** rfolco has quit IRC01:54
*** winggundamth has joined #openstack-infra01:57
openstackgerritliusheng proposed openstack-infra/irc-meetings master: Add Mogan (Baremetal Computing service) meeting  https://review.openstack.org/44734001:57
*** jamesdenton has joined #openstack-infra01:58
*** jamesdenton has quit IRC01:58
*** rfolco has joined #openstack-infra02:00
*** rfolco has quit IRC02:00
*** kzaitsev_mb has quit IRC02:00
openstackgerritLana Brindley proposed openstack-infra/infra-manual master: Adding instructions for rebasing  https://review.openstack.org/44734102:02
*** mat128 has joined #openstack-infra02:04
openstackgerritliusheng proposed openstack-infra/irc-meetings master: Add Mogan (Baremetal Computing service) meeting  https://review.openstack.org/44734002:08
*** mat128 has quit IRC02:14
*** yamamoto has joined #openstack-infra02:19
*** yamamoto has quit IRC02:19
*** mat128 has joined #openstack-infra02:21
*** EricGonczer_ has joined #openstack-infra02:22
*** mat128 has quit IRC02:25
*** yamamoto has joined #openstack-infra02:31
*** aeng has quit IRC02:43
*** EricGonczer_ has quit IRC02:50
*** dimtruck is now known as zz_dimtruck02:52
*** kzaitsev_mb has joined #openstack-infra02:55
*** baoli has quit IRC03:14
*** gildub has quit IRC03:17
*** dave-mccowan has quit IRC03:20
*** EricGonczer_ has joined #openstack-infra03:21
*** ramishra has joined #openstack-infra03:22
*** jamielennox|away is now known as jamielennox03:22
*** zhurong has joined #openstack-infra03:25
*** aeng has joined #openstack-infra03:27
*** EricGonczer_ has quit IRC03:29
*** jiaohaolin1 has left #openstack-infra03:32
*** sree has joined #openstack-infra03:33
*** kzaitsev_mb has quit IRC03:36
*** yamamoto has quit IRC03:50
*** yamamoto has joined #openstack-infra03:57
*** yamamoto has quit IRC03:59
*** yamamoto has joined #openstack-infra04:00
*** yamamoto has quit IRC04:00
*** Dinesh_Bhor has joined #openstack-infra04:08
*** psachin has joined #openstack-infra04:09
*** tphummel has joined #openstack-infra04:11
*** zhurong has quit IRC04:16
*** gongysh has quit IRC04:18
*** links has joined #openstack-infra04:20
*** ihrachys has joined #openstack-infra04:20
*** ihrachys has quit IRC04:21
*** kzaitsev_mb has joined #openstack-infra04:32
*** tphummel has quit IRC04:44
*** rwsu has joined #openstack-infra04:51
*** yamamoto has joined #openstack-infra04:55
*** zhurong has joined #openstack-infra04:59
*** yamamoto has quit IRC05:00
*** gongysh has joined #openstack-infra05:04
*** kornicameister has quit IRC05:06
*** kzaitsev_mb has quit IRC05:10
*** kornicameister has joined #openstack-infra05:13
*** zhurong has quit IRC05:14
*** saibarspeis has joined #openstack-infra05:19
*** bhavik1 has joined #openstack-infra05:21
*** anilvenkata has joined #openstack-infra05:26
*** Julien-zte has joined #openstack-infra05:28
*** nmathew has joined #openstack-infra05:32
*** saibarspeis has quit IRC05:33
*** saibarspeis has joined #openstack-infra05:34
*** vikrant has joined #openstack-infra05:34
*** ijw has quit IRC05:39
*** ijw has joined #openstack-infra05:40
*** kornicameister has quit IRC05:41
*** kornicameister has joined #openstack-infra05:43
*** claudiub has joined #openstack-infra05:46
*** dhajare has joined #openstack-infra05:46
*** jiaohaolin1 has joined #openstack-infra05:49
*** adriant has quit IRC05:53
*** gongysh has quit IRC05:58
*** pgadiya has joined #openstack-infra05:59
*** jaosorior has joined #openstack-infra06:04
*** prateek has joined #openstack-infra06:05
*** kzaitsev_mb has joined #openstack-infra06:06
*** Sukhdev has joined #openstack-infra06:07
*** nadya has joined #openstack-infra06:08
*** gongysh has joined #openstack-infra06:10
*** yamamoto has joined #openstack-infra06:11
*** nadya has quit IRC06:14
*** yamamoto has quit IRC06:18
*** nadya has joined #openstack-infra06:18
*** nadya has quit IRC06:29
*** yamahata has joined #openstack-infra06:29
*** kzaitsev_mb has quit IRC06:30
*** kornicameister has quit IRC06:41
*** 7IZAATF07 has joined #openstack-infra06:42
*** 7GHAALN3P has joined #openstack-infra06:42
*** 7GHAALN3P has quit IRC06:42
*** jiaohaolin1 has left #openstack-infra06:43
*** AJaeger has quit IRC06:43
*** kornicameister has joined #openstack-infra06:43
*** Sukhdev has quit IRC06:48
*** jaosorior has quit IRC06:53
*** sree has quit IRC06:54
*** jaosorior has joined #openstack-infra06:55
*** yujunz has joined #openstack-infra06:57
*** sree has joined #openstack-infra07:00
*** sree_ has joined #openstack-infra07:04
*** sree_ is now known as Guest8436007:05
*** sree has quit IRC07:05
*** Guest84360 has quit IRC07:08
*** florianf has joined #openstack-infra07:10
*** jaosorior has quit IRC07:14
*** jaosorior has joined #openstack-infra07:15
*** jaosorior has quit IRC07:16
*** dhajare has quit IRC07:17
*** slaweq has joined #openstack-infra07:17
*** jaosorior has joined #openstack-infra07:19
*** alexchadin has joined #openstack-infra07:19
*** andreas_s has joined #openstack-infra07:24
*** kzaitsev_mb has joined #openstack-infra07:25
*** Jeffrey4l__ has quit IRC07:33
*** esikachev has joined #openstack-infra07:34
*** dhajare has joined #openstack-infra07:34
*** Jeffrey4l__ has joined #openstack-infra07:36
*** makowals has joined #openstack-infra07:37
*** yonglihe has quit IRC07:39
*** csomerville has joined #openstack-infra07:49
*** markus_z has joined #openstack-infra07:49
openstackgerritChandan Kumar proposed openstack-infra/tripleo-ci master: [TEST JOB] test tempest-12.2.0 for Mitaka  https://review.openstack.org/44686807:49
*** ralonsoh has joined #openstack-infra07:51
*** cody-somerville has joined #openstack-infra07:51
openstackgerritChandan Kumar proposed openstack-infra/tripleo-ci master: [TEST JOB] test tempest-12.2.0 for Mitaka  https://review.openstack.org/44686807:52
*** csomerville has quit IRC07:53
*** claudiub|2 has joined #openstack-infra07:57
*** tesseract has joined #openstack-infra07:58
*** saibarspeis has quit IRC07:59
*** dhajare has quit IRC07:59
*** claudiub has quit IRC08:00
*** zzzeek has quit IRC08:00
*** matrohon has joined #openstack-infra08:00
*** zzzeek has joined #openstack-infra08:00
*** kzaitsev_mb has quit IRC08:03
*** kaisers2 has quit IRC08:04
*** shardy has joined #openstack-infra08:04
*** shardy has quit IRC08:04
*** armax has quit IRC08:05
*** ociuhandu has joined #openstack-infra08:06
*** shardy has joined #openstack-infra08:07
openstackgerritMike Kolesnik proposed openstack-infra/project-config master: Skip non-doc jobs for networking-odl doc changes  https://review.openstack.org/44740508:08
*** sshnaidm|off is now known as sshnaidm08:09
*** neiljerram has quit IRC08:11
*** jiaohaolin1 has joined #openstack-infra08:11
*** dhajare has joined #openstack-infra08:14
*** pcaruana has joined #openstack-infra08:14
*** martinkopec has joined #openstack-infra08:15
*** nadya has joined #openstack-infra08:16
*** kaisers has joined #openstack-infra08:18
*** nadya has quit IRC08:21
*** yamamoto has joined #openstack-infra08:22
*** ssbarnea has joined #openstack-infra08:25
openstackgerritMerged openstack-infra/project-config master: Enabled networking-l2gw-release to create tags  https://review.openstack.org/44727908:27
*** chem has quit IRC08:28
*** hashar has joined #openstack-infra08:31
openstackgerritMerged openstack-infra/project-config master: [I18n] accept header change support with PEP8  https://review.openstack.org/44717808:32
*** matrohon has quit IRC08:32
*** kaisers has quit IRC08:34
*** wolverineav has joined #openstack-infra08:35
openstackgerritMerged openstack-infra/project-config master: Move #openstack-opw to #openstack-outreachy  https://review.openstack.org/44713108:36
*** dingyichen has quit IRC08:39
*** wolverineav has quit IRC08:40
openstackgerritMerged openstack-infra/project-config master: telemetry: switch to default driver  https://review.openstack.org/44439008:40
openstackgerritMerged openstack-infra/project-config master: post git events to golang irc channel  https://review.openstack.org/44656908:45
openstackgerritMerged openstack-infra/project-config master: Enable Heat in Devstack for Patrole  https://review.openstack.org/44672608:45
*** wolverineav has joined #openstack-infra08:45
*** sshnaidm is now known as sshnaidm|afk08:47
*** dtantsur|afk is now known as dtantsur08:47
*** vtapia has quit IRC08:48
*** kaisers has joined #openstack-infra08:49
*** vtapia has joined #openstack-infra08:50
*** wolverineav has quit IRC08:50
*** jpich has joined #openstack-infra08:52
*** isaacb has joined #openstack-infra08:52
*** wolverineav has joined #openstack-infra08:55
*** jtomasek|afk is now known as jtomasek08:57
*** pgadiya has quit IRC08:58
*** kzaitsev_mb has joined #openstack-infra08:58
*** wolverineav has quit IRC09:00
*** rossella_s has joined #openstack-infra09:02
*** openstackgerrit has quit IRC09:02
*** yamamoto has quit IRC09:04
*** wolverineav has joined #openstack-infra09:06
*** pgadiya has joined #openstack-infra09:08
*** csomerville has joined #openstack-infra09:09
*** cody-somerville has quit IRC09:09
*** isaacb has quit IRC09:09
*** wolverineav has quit IRC09:10
*** openstackgerrit has joined #openstack-infra09:11
openstackgerritVasyl Saienko proposed openstack-infra/devstack-gate master: DNR/TMP Increase tempest concurrency to 4  https://review.openstack.org/44713609:11
*** wolverineav has joined #openstack-infra09:16
*** e0ne has joined #openstack-infra09:18
*** wolverineav has quit IRC09:20
claudiub|2fungi: hi. around?09:20
*** claudiub|2 is now known as claudiub09:20
openstackgerritzhangshuai proposed openstack-infra/project-config master: karbor: enable cinder multi-workers, difference conf of CI and devstack  https://review.openstack.org/44626809:21
*** e0ne has quit IRC09:24
*** e0ne has joined #openstack-infra09:24
*** wolverineav has joined #openstack-infra09:26
*** kzaitsev_mb has quit IRC09:26
*** alexchadin has quit IRC09:27
*** lucas-afk is now known as lucasagomes09:28
*** wolverineav has quit IRC09:30
*** Julien-zte has quit IRC09:35
*** cody-somerville has joined #openstack-infra09:35
*** neiljerram has joined #openstack-infra09:35
*** wolverineav has joined #openstack-infra09:36
*** yujunz has quit IRC09:36
*** nadya has joined #openstack-infra09:36
*** electrofelix has joined #openstack-infra09:37
*** csomerville has quit IRC09:37
*** wolverineav has quit IRC09:40
*** neil_ has joined #openstack-infra09:45
*** derekh has joined #openstack-infra09:45
*** neil_ has quit IRC09:46
*** wolverineav has joined #openstack-infra09:46
*** yamahata has quit IRC09:48
*** andreww has quit IRC09:49
*** wolverineav has quit IRC09:50
*** wolverineav has joined #openstack-infra09:56
*** mozart89 has joined #openstack-infra09:57
*** alexchadin has joined #openstack-infra09:58
*** xarses has joined #openstack-infra10:00
*** wolverineav has quit IRC10:00
*** yamamoto has joined #openstack-infra10:03
*** kornica has joined #openstack-infra10:03
*** wolverineav has joined #openstack-infra10:06
*** pbourke has quit IRC10:07
*** pbourke has joined #openstack-infra10:07
*** yamamoto has quit IRC10:07
*** liujiong has quit IRC10:09
*** sshnaidm|afk is now known as sshnaidm10:09
*** kornicameister has quit IRC10:09
*** wolverineav has quit IRC10:11
*** nicolasbock has joined #openstack-infra10:12
openstackgerrityolanda.robla proposed openstack/diskimage-builder master: Add dracut-regenerate elements  https://review.openstack.org/44704710:14
claudiubyolanda: hi.10:15
claudiubaround for a sec?10:15
yolandahi10:16
*** sree has joined #openstack-infra10:17
*** knangia has quit IRC10:21
*** yamamoto has joined #openstack-infra10:21
*** yamamoto has quit IRC10:21
*** sree has quit IRC10:21
*** panda|Zz is now known as panda10:21
claudiubyolanda: so, this merged last week: https://review.openstack.org/#/c/441810/  . Could you add me to the python-hnvclient core / release teams?10:22
*** AJaeger has joined #openstack-infra10:22
claudiubgroups: https://review.openstack.org/#/admin/groups/1726,members and https://review.openstack.org/#/admin/groups/1727,members10:22
openstackgerritMerged openstack-infra/irc-meetings master: Add Mogan (Baremetal Computing service) meeting  https://review.openstack.org/44734010:23
*** kzaitsev_mb has joined #openstack-infra10:23
openstackgerritMerged openstack-infra/irc-meetings master: Register the OpenStack Upstream Institute team meetings  https://review.openstack.org/44729110:24
*** florianf has quit IRC10:25
*** ijw has quit IRC10:25
yolandasure10:27
*** florianf has joined #openstack-infra10:27
claudiub\o/ thanks! :D10:27
*** tosky has joined #openstack-infra10:27
*** ijw has joined #openstack-infra10:27
yolandadone10:28
*** pblaho has joined #openstack-infra10:28
*** jkilpatr has quit IRC10:41
*** hashar is now known as hasharLunch10:44
*** yujunz has joined #openstack-infra10:50
*** yujunz-zte has joined #openstack-infra10:51
openstackgerritMerged openstack-infra/project-config master: ovsdbapp: Add functional job to check queue  https://review.openstack.org/44695310:53
openstackgerritVasyl Saienko proposed openstack-infra/devstack-gate master: DNR/TMP Increase tempest concurrency to 4  https://review.openstack.org/44713610:54
*** matrohon has joined #openstack-infra10:54
*** yujunz has quit IRC10:55
*** wolverineav has joined #openstack-infra10:55
*** jkilpatr has joined #openstack-infra10:58
*** wolverineav has quit IRC11:00
openstackgerritJulien Danjou proposed openstack-infra/project-config master: Remove Gnocchi doc publishing on docs.o.o  https://review.openstack.org/44745011:02
*** alexchadin has quit IRC11:04
*** psachin has quit IRC11:05
*** nmathew has quit IRC11:05
*** 7IZAATF07 has quit IRC11:07
*** bhavik1 has quit IRC11:08
*** yanyanhu_ has joined #openstack-infra11:09
*** yanyanhu_ has quit IRC11:09
*** kzaitsev_mb has quit IRC11:09
*** matrohon has quit IRC11:12
openstackgerritVitaly Gridnev proposed openstack-infra/project-config master: [sahara multinode] Specify empty config for subnode  https://review.openstack.org/44721511:15
*** aarefiev_afk is now known as aarefiev11:15
*** yujunz-zte has quit IRC11:16
*** zhurong has joined #openstack-infra11:17
*** matrohon has joined #openstack-infra11:18
*** mozart89 has quit IRC11:18
*** yamamoto has joined #openstack-infra11:22
*** mozart89 has joined #openstack-infra11:22
*** yamamoto has quit IRC11:30
*** alexchadin has joined #openstack-infra11:33
*** Jeffrey4l_ has joined #openstack-infra11:34
*** Jeffrey4l__ has quit IRC11:36
*** zhurong has quit IRC11:37
*** ldnunes has joined #openstack-infra11:40
*** yamamoto has joined #openstack-infra11:41
*** anilvenkata has quit IRC11:42
*** psachin has joined #openstack-infra11:42
*** ociuhandu has quit IRC11:45
*** winggundamth has quit IRC11:47
*** anilvenkata has joined #openstack-infra11:48
openstackgerritzhangshuai proposed openstack-infra/project-config master: karbor: specify cinder conf and karbor conf  https://review.openstack.org/44626811:58
markus_zfrickler: Are you around?11:58
*** _ari_ is now known as _ari_|gone11:59
*** aysyd has joined #openstack-infra12:00
*** esikachev has quit IRC12:02
*** vsaienko has joined #openstack-infra12:02
*** ym200 has joined #openstack-infra12:03
*** askb has quit IRC12:04
*** pgadiya has quit IRC12:06
*** kzaitsev_mb has joined #openstack-infra12:06
*** gongysh has quit IRC12:06
*** esikachev has joined #openstack-infra12:07
fricklermarkus_z: yep. regarding https://review.openstack.org/444164 I'm fine if you just change it back, if that works in your tests12:07
*** alexchadin has quit IRC12:07
*** mat128 has joined #openstack-infra12:07
*** dave-mccowan has joined #openstack-infra12:07
markus_zfrickler: Is there a way to verify it in the gate itself?12:07
fricklermarkus_z: not without merging that patch12:08
mordredat a point in the future you'll be able to12:08
mordredbut that day is not yet today12:08
markus_zfrickler: Ah, too bad. OK, I'm going to double-check my test setup and then change/vote accordingly.12:08
markus_zWill happen in the next few hours.12:08
markus_zfrickler: Thanks for having an eye on that.12:09
fricklermarkus_z: yw, I just don't like broken nv tests ;)12:09
markus_zWhat does "nv" stand for?12:10
fricklernon-voting12:10
markus_zahh, makes sense12:10
*** dprince has joined #openstack-infra12:10
*** rfolco has joined #openstack-infra12:11
*** rhallisey has joined #openstack-infra12:11
*** alexchadin has joined #openstack-infra12:12
*** wolverineav has joined #openstack-infra12:13
*** wolverineav has quit IRC12:13
*** wolverineav has joined #openstack-infra12:13
*** wolverin_ has joined #openstack-infra12:15
*** wolverineav has quit IRC12:18
*** pradk has joined #openstack-infra12:18
*** _Cyclone_ has joined #openstack-infra12:25
*** yujunz has joined #openstack-infra12:26
*** matrohon has quit IRC12:28
*** zul has joined #openstack-infra12:29
*** ociuhandu has joined #openstack-infra12:31
*** matrohon has joined #openstack-infra12:31
*** ralonsoh_ has joined #openstack-infra12:34
*** yujunz has quit IRC12:35
*** kaisers has quit IRC12:36
*** jcoufal has joined #openstack-infra12:37
*** yujunz has joined #openstack-infra12:37
*** kgiusti has joined #openstack-infra12:37
*** yujunz has quit IRC12:39
*** yujunz-zte has joined #openstack-infra12:39
*** kzaitsev_mb has quit IRC12:42
*** shardy is now known as shardy_lunch12:44
*** rlandy has joined #openstack-infra12:45
*** esikachev has quit IRC12:47
*** esikachev has joined #openstack-infra12:48
*** kaisers has joined #openstack-infra12:49
*** mdrabe has joined #openstack-infra12:50
openstackgerrityolanda.robla proposed openstack/diskimage-builder master: Add dracut-regenerate elements  https://review.openstack.org/44704712:52
*** yuvalb has quit IRC12:53
*** trown|outtypewww is now known as trown12:53
*** yuvalb has joined #openstack-infra12:54
*** yamamoto has quit IRC12:55
openstackgerritJulien Danjou proposed openstack-infra/project-config master: Remove Gnocchi doc publishing on docs.o.o  https://review.openstack.org/44745012:56
*** mhickey has joined #openstack-infra12:56
*** mriedem has joined #openstack-infra12:56
*** eroux has joined #openstack-infra12:58
*** gongysh has joined #openstack-infra12:58
openstackgerritDavid Moreau Simard proposed openstack-infra/system-config master: Add support for mirroring the new Pike UCA repos  https://review.openstack.org/44747812:58
dmsimardmwhahaha: ^12:59
*** ihrachys has joined #openstack-infra13:01
*** markvoelker has joined #openstack-infra13:01
*** gouthamr has joined #openstack-infra13:07
*** jaypipes has joined #openstack-infra13:08
mat128dmsimard: you're missing a single quote, btw :)13:10
*** links has quit IRC13:11
*** kaisers has quit IRC13:12
dmsimardmat128: oh hai. Doh.13:13
mat128:)13:13
*** dtantsur is now known as dtantsur|brb13:13
*** Julien-zte has joined #openstack-infra13:14
openstackgerritDavid Moreau Simard proposed openstack-infra/system-config master: Add support for mirroring the new Pike UCA repos  https://review.openstack.org/44747813:14
*** matt-borland has joined #openstack-infra13:19
*** Goneri has joined #openstack-infra13:21
openstackgerritEmilien Macchi proposed openstack-infra/system-config master: mirros: include UCA packages for Pike  https://review.openstack.org/44749013:21
mnaserEmilienM, meet dmsimard.  dmsimard, meet EmilienM ^13:22
mnaser:P13:22
dmsimardHi EmilienM I'm dmsimard13:22
EmilienMlol13:23
mnaserthe openstack hive mind is in play13:23
EmilienMmine is better13:23
dmsimardmnaser: EmilienM and I are going through puppet-openstack reviews at the same time it seems :)13:23
mnaser:P13:23
EmilienMyeah13:23
*** Sukhdev has joined #openstack-infra13:25
*** shardy_lunch is now known as shardy13:28
*** florianf has quit IRC13:29
*** kaisers has joined #openstack-infra13:30
*** jamesdenton has joined #openstack-infra13:31
*** yujunz-zte has left #openstack-infra13:32
openstackgerritMarkus Zoeller (markus_z) proposed openstack-infra/project-config master: Fix tempest-dsvm-neutron-multinode-full job VNC console settings  https://review.openstack.org/44416413:34
*** florianf has joined #openstack-infra13:36
*** kzaitsev_mb has joined #openstack-infra13:37
*** baoli has joined #openstack-infra13:37
markus_zfrickler: ^ let's try it this way. https://bugs.launchpad.net/nova/+bug/1669468 has a logstash query which will then show the impact.13:37
openstackLaunchpad bug 1669468 in OpenStack Compute (nova) "tempest.api.compute.servers.test_novnc.NoVNCConsoleTestJSON.test_novnc fails intermittently in neutron multinode nv job" [Medium,In progress] - Assigned to Markus Zoeller (markus_z) (mzoeller)13:37
*** dprince has quit IRC13:37
openstackgerritDavid Shrewsbury proposed openstack-infra/nodepool feature/zuulv3: Unlock request if it disappears  https://review.openstack.org/44750113:37
*** Sukhdev has quit IRC13:39
*** dprince has joined #openstack-infra13:39
*** Sukhdev has joined #openstack-infra13:40
*** ralonsoh_ has quit IRC13:40
fricklermarkus_z: +113:40
markus_zthanks13:40
*** Hal has joined #openstack-infra13:41
asettleHey infra admins - any chance I could get admin access to the openstack-doc chan? :)13:42
*** eharney has joined #openstack-infra13:42
*** chem has joined #openstack-infra13:42
*** clenimar has quit IRC13:43
fungiasettle: let's find out!13:44
asettlefungi: hooray! \o/13:44
asettleI promise I'm super trust worthy with such responsibility13:46
fungi13:45 freenode -- ChanServ (ChanServ@services.): Flags +Aeortv were set on asettle in #openstack-doc.13:46
* asettle goes off to create carnage 13:46
jamespagequick question - whats the machine spec on an instance used to run unit tests for projects in the gate?13:47
fungijamespage: https://docs.openstack.org/infra/manual/testing.html13:47
*** hongbin has joined #openstack-infra13:47
fungijamespage: if you have specific questions not answered there, we can certainly add to the list13:48
jamespagefungi: thankyou - looking now13:48
jamespagefungi: hmm - yeah >= 8GB of RAM - I'm working on a theory why we see odd neutron + dependency related unit test failures in launchpad builders for Ubuntu13:49
jamespagethey only get 4GB or RAM - might be memory pressure related but not sure yet13:49
fricklerjamespage: thats not unlikely, we see lots of errors hitting the 8GB limit for complex tests13:50
fricklerjamespage: not unit tests, though13:50
*** rhallisey has quit IRC13:50
mnaserif i was to do a little write up about proposing/requesting a persistent slave and the use case for it.. where would be the best way to go about this?13:51
fungijamespage: i remember when they ran successfully on instances with 2gb ram. when we combined all jobs into one pool and needed 8gb for full devstack deployments, lots of teams unit testing slowly expanded to fill the available memory as there was no longer any backpressure there13:51
*** mat128 has quit IRC13:51
AJaegermnaser: we have an infra-specs repository for such proposals13:51
openstackgerritMerged openstack-infra/zuul master: Update hacking dependency  https://review.openstack.org/44253313:52
fungimnaser: an infra spec. example: http://specs.openstack.org/openstack-infra/infra-specs/specs/artifact-signing.html13:52
jamespagefungi, frickler: typical example - https://launchpadlibrarian.net/311577782/buildlog_ubuntu-zesty-amd64.vmware-nsx_10.0.1~git20170317.bfa48c0-0ubuntu1_BUILDING.txt.gz13:52
fungithat's the spec where i described the case and implementation for the signing worker13:52
jamespageits always some sort of missing db table type issue in sqlite13:53
jamespagewhich for neutron unit tests is in memory13:53
*** gcb has joined #openstack-infra13:53
mnaserfungi: alright, i'll try to draft up something.  /me has never written a spec for openstack before13:53
fungimnaser: we have a template spec in the openstack-infra/infra-specs repo you can copy into the approved specs directory and just fill in the blanks13:53
jamespagebuilder spec is 4G ram/4G swap and 4 cores; so going to try repro on an instance of that spec13:53
*** armax has joined #openstack-infra13:53
*** lucasagomes is now known as lucas-hungry13:53
fungimnaser: then push it for review just like any other repo13:54
mnaserokay cool, i'll draft up something13:54
fungimnaser: the readme in that repo gives you quick steps13:55
*** yamamoto has joined #openstack-infra13:55
mnaserfungi awesome, i'll publish a draft, go over it with some kolla folks and then once we have that in place, we'll remove the draft flag13:55
fungis/draft/wip/ i assume (gerrit also has a "draft" feature which we explicitly disable)13:56
*** rhallisey has joined #openstack-infra13:57
*** gcb has quit IRC13:57
AJaegerfungi, looking at global IRC access list: It includes Sam-I-am, lifeless, maffulli - do you want to remove some of them?13:58
fungiAJaeger: lifeless and reed are both in channel here, so if they think they want to continue helping police irc channel issues i'm fine leaving them in the list13:59
AJaegeragreed14:01
*** panda is now known as panda|off14:01
AJaegerSo, should I propose removing Sam-I-am? - and copy him on the review?14:01
*** dprince has quit IRC14:01
*** dprince has joined #openstack-infra14:03
fungii'm looking to see if he's still around in any capacity14:03
*** yamamoto has quit IRC14:04
fricklerjamespage: I'm not sure whether testing on sqlite is supported at all anymore for that project. they do setup mysql for their upstream tests: https://github.com/openstack/vmware-nsx/blob/master/tools/test-setup.sh14:05
fungiaccording to http://stackalytics.com/?user_id=ionosphere80&metric=marks&release=all his code review activity ended in november14:05
*** knangia has joined #openstack-infra14:05
*** hasharLunch is now known as hashar14:05
jamespagefrickler: that covers the mysql db migration testing - I don't think that's used for the bulk of the unit tests14:05
jamespagefrickler: fwiw I have reached out to that team for help :-)14:06
fungiAJaeger: i guess he was primarily a docs contributor so you would know better than i if he's moved on i guess14:06
AJaegerfungi, he has moved on.14:06
openstackgerritAndreas Jaeger proposed openstack-infra/project-config master: Remove Sam-I-am from accessbot  https://review.openstack.org/44751114:07
fungiAJaeger: in that case, sure let's take him out of the accessbot config (that doesn't remove him from any existing channels however)14:07
fricklerfungi: http://lists.openstack.org/pipermail/openstack-dev/2016-December/109302.html14:07
openstackgerritAndreas Jaeger proposed openstack-infra/project-config master: Remove Sam-I-am from accessbot  https://review.openstack.org/44751114:09
fungithanks frickler14:09
AJaegerfrickler: ARgh, I now see that I added the wrong link to my commit message...14:09
AJaegerfungi, I'll give him a week to review this and +1/-1 - have WIP for now.14:10
fungithanks!14:11
*** kzaitsev_mb has quit IRC14:12
*** dprince has quit IRC14:12
*** eroux has quit IRC14:12
openstackgerritSagi Shnaidman proposed openstack-infra/tripleo-ci master: DONT REVIEW: test consistent repo  https://review.openstack.org/44751414:14
openstackgerritSagi Shnaidman proposed openstack-infra/tripleo-ci master: DONT REVIEW: test consistent repo  https://review.openstack.org/44751414:15
*** markmcd has quit IRC14:15
openstackgerritSagi Shnaidman proposed openstack-infra/tripleo-ci master: DONT REVIEW: test consistent repo  https://review.openstack.org/44751414:15
*** dprince has joined #openstack-infra14:17
*** gcb_ has joined #openstack-infra14:17
gcb_infra team: I would like to delete a branch for oslo.cache, what should I do ?14:18
*** markmcd has joined #openstack-infra14:19
*** eroux has joined #openstack-infra14:21
fungigcb_: abandon any open changes for that branch, consider tagging the final state of teh branch if anyone's likely to ever want to check it out in the future, and then let us know the name of the repo and branch so we can do the deletion14:22
*** annegentle has joined #openstack-infra14:22
gcb_oslo.cache with branch remotes/gerrit/feature/keystone-initial-import14:23
*** burgerk has joined #openstack-infra14:23
gcb_fugi: there is no open change for that branch, I think we can just delete it14:24
*** ijw has quit IRC14:25
*** EricGonczer_ has joined #openstack-infra14:26
*** gordc has joined #openstack-infra14:26
openstackgerritEmilien Macchi proposed openstack-infra/tripleo-ci master: DONT REVIEW: test consistent repo  https://review.openstack.org/44752214:27
*** ijw has joined #openstack-infra14:28
*** cody-somerville has quit IRC14:29
*** vikrant has quit IRC14:30
*** zz_dimtruck is now known as dimtruck14:30
*** rbrndt has joined #openstack-infra14:31
*** nadya has quit IRC14:33
*** EricGonc_ has joined #openstack-infra14:33
*** sekelso has joined #openstack-infra14:33
mnaserwelp, draft workflow is disabled on infra-specs?14:34
mnaserguess ill make noise >:(14:34
openstackgerritMohammed Naser proposed openstack-infra/infra-specs master: DockerHub Image Publisher  https://review.openstack.org/44752414:34
*** sekelso is now known as skelso14:34
*** EricGonczer_ has quit IRC14:34
openstackgerritVasyl Saienko proposed openstack-infra/project-config master: Increase concurrency for ironic jobs to 4  https://review.openstack.org/44752514:35
*** matt-borland has quit IRC14:36
*** matt-borland has joined #openstack-infra14:37
AJaegermnaser: as fungi said earlier: Use WIP, OpenStack gerrit has draft workflow disabled globally.14:38
mnaseraaah14:38
mnasersorry i missed that14:38
sdaguemnaser: draft was user confusing, people did not realize that draft makes reviews private14:38
*** tphummel has joined #openstack-infra14:38
mnasersdague didnt know that so i can see why its confusing!14:39
sdagueyeh14:39
openstackgerritMohammed Naser proposed openstack-infra/infra-specs master: WIP: DockerHub Image Publisher  https://review.openstack.org/44752414:39
sdagueit caused some very weird situations where people took over reviews that were public, then they became private and it was very confusing :)14:39
fungisell, semi-private. only displayed it to subscribed reviewers, but didn't hide it from git replication14:40
fungis/sell/well/14:40
sdaguefungi: right14:40
*** isq has quit IRC14:40
fungialso created some odd issues for ci since you could have a patch series with one public change parented to a draft change14:40
fungiso the ci system, unable to find the draft through the gerrit api (due to not being subscribed) couldn't check out the child change to test it14:41
*** diltram has quit IRC14:44
*** dtantsur|brb is now known as dtantsur14:44
*** Goneri has quit IRC14:44
*** witek has joined #openstack-infra14:44
witekhello14:45
witekwhich job should I add to projects.yml to publish the tarball for stable branches to https://tarballs.openstack.org14:46
witek?14:46
*** florianf has quit IRC14:47
fungiwitek: {name}-branch-tarball14:47
*** yamahata has joined #openstack-infra14:47
mnaserAJaeger do i list it under "approved design" in index.rst (because technically it will be on merge?)14:48
fungimnaser: correct14:48
mnaserthank you fungi14:48
*** florianf has joined #openstack-infra14:48
*** martinkopec has quit IRC14:48
witekfungi: thanks14:48
fungimnaser: maybe our readme should be explicit about that, we seem to get the question frequently14:48
*** marst has quit IRC14:48
mnaseryeah, feels a bit weird to just throw in in approved :p14:49
gcb_fungi: please delete branch remotes/gerrit/feature/keystone-initial-import for oslo.cache , thanks. it blocks oslo release new libs in https://review.openstack.org/44743714:49
openstackgerritMohammed Naser proposed openstack-infra/infra-specs master: WIP: DockerHub Image Publisher  https://review.openstack.org/44752414:49
*** martinkopec has joined #openstack-infra14:50
openstackgerritJulien Danjou proposed openstack-infra/project-config master: Remove Gnocchi doc publishing on docs.o.o  https://review.openstack.org/44745014:50
*** armax has quit IRC14:53
witekfungi: python-jobs group includes '{name}-branch-tarball', monasca-api project lists python-jobs, still there is no tarball monasca-api-stable-ocata.tar.gz14:53
*** lucas-hungry is now known as lucasagomes14:56
*** marst has joined #openstack-infra14:56
*** marst has quit IRC14:56
*** marst has joined #openstack-infra14:56
*** dtantsur is now known as dtantsur|mtg14:57
openstackgerritCorey O'Brien proposed openstack/diskimage-builder master: Don't unmount missing paths  https://review.openstack.org/44662114:58
*** rbrndt has quit IRC14:59
*** xarses_ has joined #openstack-infra14:59
*** rbrndt has joined #openstack-infra15:00
*** rbrndt has quit IRC15:00
openstackgerritDavid Shrewsbury proposed openstack-infra/nodepool feature/zuulv3: Revert "Refactor nodepool apps into base app"  https://review.openstack.org/44753415:00
*** annegentle has quit IRC15:00
*** andreww has joined #openstack-infra15:01
fungiwitek: because in zuul/layout.yaml monasca-api doesn't use the python-jobs template but also doesn't explicitly add monasca-api-branch-tarball in its post pipeline15:01
*** annegentle has joined #openstack-infra15:02
*** rbrndt has joined #openstack-infra15:02
witekthank you again :)15:02
fungigcb_: i've just now deleted the feature/keystone-initial-import branch of openstack/oslo.cache, previously at 864dfacced6ce1a86cb08fbd7a257b33b2c59e1d15:03
*** asselin has joined #openstack-infra15:03
AJaegerwitek: you define in python/jobs the jobs you could run - and trigger running them in zuul/layout.15:04
*** alexchadin has quit IRC15:04
*** xarses_ has quit IRC15:04
gcb_fungi, thanks15:04
*** skelso has quit IRC15:05
*** skelso has joined #openstack-infra15:06
witekAJaeger: yes, you have written that to me already a couple of times, but it seems I'm reluctant to learn15:06
AJaegerfungi, project-config change fails with "No matching distribution found for jenkins-job-builder==1.6.1" - see http://logs.openstack.org/11/447511/2/check/gate-project-config-layout/d52f126/console.html15:06
*** diltram has joined #openstack-infra15:07
fungioh that's a fun one15:07
AJaegerwitek: shall I write it in German for you? ;)15:07
pabelangerHmm, looks like we are having some networking issues with osic-cloud115:07
pabelanger2017-03-20 14:45:41.210959 | W: Failed to fetch http://mirror.regionone.osic-cloud1.openstack.org/ubuntu/dists/xenial/InRelease  Could not connect to mirror.regionone.osic-cloud1.openstack.org:80 (172.99.106.183), connection timed out [IP: 172.99.106.183 80]15:08
AJaegerfungi, wrong error line, sorry.15:08
*** kzaitsev_mb has joined #openstack-infra15:08
AJaegerfungi, pabelanger "fter connection broken by 'ReadTimeoutError("HTTPConnectionPool(host='mirror.iad.rax.openstack.org', port=80): Read timed out"15:08
witekI will make a big note on my display :)15:08
pabelangerAJaeger: looks like a network issue15:08
AJaegeryeah ;(15:09
* AJaeger needs a break and hope the network heals itself in the meantime ;)15:09
*** asselin_ has joined #openstack-infra15:10
fungipabelanger: AJaeger: wow, connection issues to both osic and rax-iad mirrors at the same time?15:10
openstackgerritWitold Bedyk proposed openstack-infra/project-config master: Add python-jobs for monasca-api  https://review.openstack.org/44753915:10
pabelanger2017-03-20 14:31:33.330433 |   Retrying (Retry(total=4, connect=None, read=None, redirect=None)) after connection broken by 'ReadTimeoutError("HTTPConnectionPool(host='mirror.mtl01.internap.openstack.org', port=80): Read timed out. (read timeout=60.0)",)': /pypi/packages/0c/85/f495b58b2b0ac907def07385219e9747b75840fa01280f228546a4a5ad7f/testrepository-0.0.20.tar.gz15:10
EmilienMhttp://mirror.regionone.osic-cloud1.openstack.org/centos/7/os/x86_64/repodata/repomd.xml: [Errno 12] Timeout on http://mirror.regionone.osic-cloud1.openstack.org/centos/7/os/x86_64/repodata/repomd.xml: (28, 'Operation too slow. Less than 1000 bytes/sec transferred the last 30 seconds')15:10
pabelangerand internap15:10
fungiummm...15:10
pabelangerfungi: something is going on15:10
EmilienMhello, we have timeouts in tripleo Ci15:10
EmilienMI guess you're aware of it if I read correctly the backlog15:11
pabelangerEmilienM: yes, we are looking into it15:11
EmilienMack15:11
EmilienMthanks15:11
fungivery suddenly aware, but aware15:11
fungijust starting to dig into possible causes15:11
clarkbdfw seems fine if I browse it by hand (and its the one adjacent to osic iirc)15:11
pabelangerYa, internap seems okay atm15:12
*** tphummel has quit IRC15:12
fungiafs01.dfw is dipping into swap15:12
clarkbinteresting15:12
*** asselin has quit IRC15:12
fungicache memory utilization on afs02.dfw also just dropped by half15:13
*** martinkopec has quit IRC15:14
clarkbdafileserver using all the memory on afs0115:14
pabelangerhttp://cacti.openstack.org/cacti/graph.php?action=view&local_graph_id=2359&rra_id=all15:14
pabelangerconfirms the growing swap15:14
pabelangerinterestingly, we had a swap growth back in Nov15:14
pabelangernot sure what we did there15:14
*** asselin has joined #openstack-infra15:14
pabelangerpossible memory leak? http://cacti.openstack.org/cacti/graph.php?action=view&local_graph_id=2357&rra_id=all15:15
fungisteady (nearly linear) memory utilization growth since beginning of december15:15
fungijust now starting to result in significant memory pressure, i think15:15
fungimaybe we need to swap primary to 02 and restart something on 01?15:16
fungijeblair: ^ if you're around15:16
openstackgerritMonty Taylor proposed openstack-infra/shade master: Cleanup some workarounds for old OCC versions  https://review.openstack.org/44754215:16
openstackgerritMonty Taylor proposed openstack-infra/shade master: Get the ball rolling on magnumclient  https://review.openstack.org/44754315:16
openstackgerritMonty Taylor proposed openstack-infra/shade master: Add normalization for cluster templates  https://review.openstack.org/44754415:16
openstackgerritMonty Taylor proposed openstack-infra/shade master: RESTify cluster template tests  https://review.openstack.org/44754515:16
openstackgerritMonty Taylor proposed openstack-infra/shade master: Convert magnum service to requests_mock  https://review.openstack.org/44754615:16
*** jamielennox has quit IRC15:17
clarkbfungi: I'm reading the docs on dafileserver and translating our command flags to understand how much cache memory it thinks it is allowed to use15:17
fungithough it looks like it previously grew by roughly the same speed previously, topping out in late october and then ran like that for over a month15:17
clarkb-vc 1200 means we can cache 1200 volumes in emmory I think15:18
fungiswap graph shows that it continued to ramp up into swap throughout november, similar to what we've seen over the past few days15:18
*** asselin__ has joined #openstack-infra15:19
clarkband since we have fewer than 1200 volumes and our volumes in aggregate use more than a terabyte of memory I think we are effectively unbounded on our cache growth (relative to 8GB host memory)15:19
fungistill all very linear growth15:19
fungii don't remember major issues with the mirrors all through november15:19
*** asselin has quit IRC15:19
*** ym200 has quit IRC15:20
openstackgerritTobias Henkel proposed openstack-infra/zuul feature/zuulv3: Change mutex to counting semaphore  https://review.openstack.org/44256315:20
clarkbeven scaling it back to 300 (which is for a small site according to docs) I think we have this problem15:21
fungiload average on the osic mirror spiked massively from 14:00 to around 14:4515:21
fungito the point where it looks like we lost a lot of snmp polls15:21
fungioh, yeah, this is fun... http://cacti.openstack.org/cacti/graph.php?action=view&local_graph_id=4965&rra_id=all (full pipe!)15:22
*** ijw has quit IRC15:23
fungii'm starting to think someone (probably us) ddos'd all our mirrors15:23
fungii don't see any obvious spikes on our ci graphs correlated to that 45-minute window though15:24
*** ijw has joined #openstack-infra15:24
clarkblooks like it happened a few weeks ago tii15:24
clarkbthe graph hole due to lack of snmp responses15:24
fungisimilar behaviot on the rax-iad mirror too15:25
*** ramishra has quit IRC15:25
fungithough on that one it looks less like we maxxed out networking, more like it was unable to serve any daya15:25
fungidata15:25
*** martinkopec has joined #openstack-infra15:26
pabelangerWe spiked to 120M in internap too15:26
fungiso maybe this is afs backend related, and the load average spikes are apache workers piling up waiting for disk15:26
*** asselin__ has quit IRC15:26
pabelangerand load also15:26
fungiand then the spike at the end of the quiet period is all of them starting to respond at once15:26
pabelangerfungi: seems common on our mirrors15:26
*** jkilpatr has quit IRC15:27
*** baoli has quit IRC15:28
*** jkilpatr has joined #openstack-infra15:28
*** prateek has quit IRC15:28
fungiafs01.dfw disk i/o graphs show some notable spikes for various block devices right at 14:0015:29
clarkbfurther reading on the other dafileserver options, the flags we set are all fairly small. Number of 16 byte structures for callbacks is 65k, and number of 100 byte structures for cached vnodes is 1200, number of 292 bytes tructures for large vnodes is 1200, number of 2092 byte structures for directory blokcs is 240, and then we set -vc to 1200 which means you can cache 1200 volumes in memory before15:29
clarkbneeding to retrive from disk15:29
fungiand there's a notable bump in swap usage right at 14:00 as well15:30
*** tonytan4ever has joined #openstack-infra15:30
*** baoli has joined #openstack-infra15:30
*** ym200 has joined #openstack-infra15:30
jeblairbut not much swap activity15:31
jeblairso based on the graph from november, maybe not too worrying?15:32
*** beekneemech is now known as bnemec15:32
*** dhajare has quit IRC15:32
*** chlong has joined #openstack-infra15:33
*** armax has joined #openstack-infra15:33
*** kzaitsev_mb has quit IRC15:35
fungiright, i expect that nump in swap utilization is a symptom of something, not a cause of the witnessed behavior15:35
jeblairthe kernel log on osic mirror says15:35
fungis/nump/bump/15:35
jeblairMar 20 14:45:37 mirror kernel: [13861203.780960] afs: Waiting for busy volume 536870932 () in cell openstack.org15:35
jeblairit had fairly regular reports of that from 14:00 through 14:4515:36
*** baoli has quit IRC15:36
jeblairthat's mirror.pypi.readonly15:36
dmsimardmordred: oi, fyi -- those empty data warnings from ara generate commands are going to be filtered by default in the next release (as to alleviate confusion into thinking it was an actual failure)15:37
fungipronounced spike to 6k read ops for /dev/xvdf at 14:00 too15:38
clarkbjeblair: and busy calls are related to the -busyat flag?15:38
*** baoli has joined #openstack-infra15:38
clarkbwe are busyat 600 incompling RPCs15:38
jeblairmirror.pypi.readonly is in the middle of a release, so it's only being served by dfw02 right now15:38
clarkbs/incompling/incoming/15:38
pabelangerdmsimard: Oh, speaking of ARA, we should think about adding bootstrapcdn support for bootstrap things. This avoid the need to copy the static folder each time we run ARA on our jobs, duplicating data on logs.o.o15:41
dmsimardpabelanger: I've thought about that. I've submitted patches to logs.o.o so that it's able to serve ARA 100% from gzipped content so those are infinitely small (compared to other things from other projects, atleast)15:44
*** jaosorior has quit IRC15:45
*** beslemon has joined #openstack-infra15:45
*** matrohon has quit IRC15:46
pabelangerdmsimard: right, I think a flag to use CND would be valid too, then we could just delete the static folder on the job, before we copy contents. I don't have any data on how many people are looking at ARA we are hosting on logs.o.o, but suspect the CDN network could handle the traffic.15:46
dmsimardIn the context of packaging ARA (for distros) I will certainly have to move away from bundling those static deps in-tree anyway. Relying on python-xstatic or CDN are options.15:46
jeblairMon Mar 20 13:57:49 2017 Scheduling salvage for volume 536870932 on part /vicepa over FSSYNC15:47
fungioh, there we go15:48
fungii wonder why it needed to try to salvage15:48
*** Goneri has joined #openstack-infra15:48
jeblair03/20/2017 14:45:35 Salvaged mirror.pypi (536870931): 2151915 files, 555101483 blocks15:48
jeblairthat matches up with the downtime15:49
jeblairthese messages showed up a lot before the salvage:15:49
jeblairMon Mar 20 13:57:49 2017 SYNC_ask: negative response on circuit 'FSSYNC'15:49
jeblairMon Mar 20 13:57:49 2017 FSYNC_askfs: FSSYNC request denied for reason=6553715:49
jeblairMon Mar 20 13:57:49 2017 VAttachVolume: file server says volume 536870932 is salvaging15:49
*** matrohon has joined #openstack-infra15:50
openstackgerrityolanda.robla proposed openstack/diskimage-builder master: Add dracut-regenerate elements  https://review.openstack.org/44704715:50
fungidoesn't seem to have been scsi timeouts or similar errors (otherwise those would be present in dmesg output)15:50
*** gongysh has quit IRC15:50
jeblairoh, i guess those messages were actually after it started salvaging15:50
jeblair03/20/2017 13:56:07 dispatching child to salvage volume 536870932...15:51
jeblair03/20/2017 13:56:07 dispatching child to salvage volume 536870931...15:51
jeblairthat's when it started15:51
*** links has joined #openstack-infra15:51
*** psachin has quit IRC15:51
jeblairi don't know why it decided to do that15:52
openstackgerrityolanda.robla proposed openstack/diskimage-builder master: Always execute autorelabel  https://review.openstack.org/44707615:53
jeblairMon Mar 20 13:56:07 2017 Scheduling salvage for volume 536870932 on part /vicepa over SALVSYNC15:54
*** links has quit IRC15:55
clarkbreading man page for salvager, it is salvaging corrupted volumes to return them to a happy state15:55
clarkb"restores internal consistency" is the terminology used there15:55
*** links has joined #openstack-infra15:56
*** annegentle has quit IRC15:58
*** jaosorior has joined #openstack-infra15:59
jeblairthe ro volume is being recloned, btw.15:59
jeblairthat will probably take a while15:59
jeblair(and might timeout)16:00
*** annegentle has joined #openstack-infra16:00
*** trown is now known as trown|outtypewww16:00
*** markus_z has quit IRC16:01
*** rlandy is now known as rlandy|brb16:01
jeblairi don't know what the triggering event was :/.  i guess we should just check back in on it in a while to make sure the re-clone is okay.16:02
fungiyep, i hunted high and low... no rackspace tickets, no disruption reported on their status page (except for load balancer services), no ongoing maintenance, nothing in system logs indicating disk issues...16:04
*** armax has quit IRC16:05
*** links has quit IRC16:05
*** dtantsur|mtg is now known as dtantsur16:11
clarkbjeblair: fungi so we don't think the high memory use by dafileserver is related? Is that something we should be attempting to address separately? the -vc 1200 config option seems to be really odd from a size your server's cache perspective16:12
fungiclarkb: given that we don't have a smoking gun it's hard to say for sure it's unrelated, but we did seem to survive all the way through november that way16:13
*** tphummel has joined #openstack-infra16:13
fungiand also i'd be surprised if starting to exercise swap memory results in filesystem corruption sufficient to cause the salvager to kick in16:14
fungiso i'm operating under the assumption that they're probably separate and unrelated16:15
clarkbya swapping should just slow things down not flip bits16:15
*** tesseract has quit IRC16:15
fungianyway, that aside, it does still seem like a suboptimal performance situation (there's system cpu spikes up to 1 ful core's worth on the graphs when swap starts to come into play, suggesting we may see some minor performance drops)16:16
*** Sukhdev has quit IRC16:16
*** jkilpatr has quit IRC16:16
clarkbjeblair: is there an implicit volume size assumption that I am missing somewhere? like maybe its means cache volume metadata not volume contents?16:17
fungithankfully end consumers of the data are accessing it through a separate cache layer, which is probably why we don't see actual performance issues16:17
*** jamielennox has joined #openstack-infra16:19
*** arxcruz has quit IRC16:19
openstackgerritMonty Taylor proposed openstack-infra/shade master: Cleanup some workarounds for old OCC versions  https://review.openstack.org/44754216:21
openstackgerritMonty Taylor proposed openstack-infra/shade master: Get the ball rolling on magnumclient  https://review.openstack.org/44754316:21
openstackgerritMonty Taylor proposed openstack-infra/shade master: Add normalization for cluster templates  https://review.openstack.org/44754416:21
openstackgerritMonty Taylor proposed openstack-infra/shade master: RESTify cluster template tests  https://review.openstack.org/44754516:21
openstackgerritMonty Taylor proposed openstack-infra/shade master: Convert magnum service to requests_mock  https://review.openstack.org/44754616:21
openstackgerritMonty Taylor proposed openstack-infra/shade master: Get rid of magnumclient dependency  https://review.openstack.org/44757816:21
*** jamielennox is now known as jamielennox|away16:21
jeblairclarkb: i think that's volume metadata16:24
clarkbjeblair: ok, if I did my maths right the other caches come out to like 2MB so I think the bulk of the memory consumption there must be related to -vc (or its just "transient" file transfer memory use)16:25
*** mozart89 has quit IRC16:26
fungitraffic graph outbound on afs01.dfw eth0 definitely shows the resync underway16:26
*** pcaruana has quit IRC16:27
openstackgerritMonty Taylor proposed openstack-infra/shade master: Use data when the data has a non-json content type  https://review.openstack.org/44758116:28
*** Sukhdev has joined #openstack-infra16:29
*** jkilpatr has joined #openstack-infra16:30
openstackgerritMonty Taylor proposed openstack-infra/shade master: Use data when the data has a non-json content type  https://review.openstack.org/44758116:30
*** adrianofr_ has joined #openstack-infra16:31
*** yamahata has quit IRC16:31
*** kzaitsev_mb has joined #openstack-infra16:31
asselin_can any puppet cores review: https://review.openstack.org/#/c/439796/ & https://review.openstack.org/#/c/263021/ ?16:32
*** xyang1 has joined #openstack-infra16:37
*** nadya has joined #openstack-infra16:44
clarkbis ~$USER/foo/path something that real bourne shell will expand? or is that a bashism?16:45
*** Julien-zte has quit IRC16:48
*** MasterOfBugs has joined #openstack-infra16:49
*** andreas_s has quit IRC16:49
*** jaosorior has quit IRC16:50
*** jaosorior has joined #openstack-infra16:52
*** Apoorva has joined #openstack-infra16:52
*** hashar is now known as hasharAway16:59
openstackgerritPaul Belanger proposed openstack-infra/nodepool feature/zuulv3: Remove SSH support from nodepool  https://review.openstack.org/44668316:59
*** Sukhdev has quit IRC17:01
*** martinkopec has quit IRC17:02
*** rlandy|brb is now known as rlandy17:02
*** esikachev has quit IRC17:04
electrofelixzxiiro: hoping to get back to reviewing a bit more consistently for jjb and fixing what I think is the last remaining concern before v2.0.0 release17:04
electrofelixfungi: wonder if you could tag the latest master for JJB with 2.0.0.0b2 as it contains a fix around use of pbr for anyone testing the v2 beta release, I know someone poked me about it last week17:06
openstackgerritEmilien Macchi proposed openstack-infra/project-config master: tripleo: exclude reno jobs before Ocata  https://review.openstack.org/44760217:06
zxiiroelectrofelix: sounds good. If you get a chance to look at the view template patch I'm stuck on how to add a new test type. Not sure how well you understand the test component of jjb but maybe you can point me in the right direction.17:06
clarkbelectrofelix: assuming I can cut a release based on gerrit acls I can do that for you shortly17:07
clarkbelectrofelix: tag 161473d1b731b0fd84abf99f8f158383d21c83a0 as 2.0.0.0b2 ?17:07
fungithanks clarkb!17:08
electrofelixclarkb: yep, that woudl work17:09
electrofelixthanks!17:09
*** anilvenkata has quit IRC17:10
clarkbok will be a minute while I get my key loaded but will ping you when its pushed17:10
fungiclarkb: why ~$USER rather than just ~ ? since $USER expands to the calling user they should do the same thing, right?17:11
fungianyway it seems to be supported by dash, at least17:11
fungii don't feel like installing posh to see what happens there17:11
*** annegentle has quit IRC17:12
*** esikachev has joined #openstack-infra17:12
clarkbfungi: sorry I meant $USER in the sense of ~root or ~clarkb not the actual USER var (sorry that was confusing)17:12
clarkbfungi: in this case its a script run by root against arbitrary system user homedir17:13
*** mat128 has joined #openstack-infra17:13
clarkband needs to support cirros but also ubuntu. Trying to make it generic enough it should run everywhere and find the users home dir17:13
fungioh, sure that also seems to work in dash at least17:13
clarkbcool17:13
fungidebian uses dash as its /bin/sh so easy for me to test17:14
*** matrohon has quit IRC17:14
clarkbelectrofelix: ok pushed17:15
*** yamahata has joined #openstack-infra17:17
*** esikache1 has joined #openstack-infra17:19
*** kzaitsev_mb has quit IRC17:19
*** sekelso has joined #openstack-infra17:20
openstackgerritAlex Schultz proposed openstack-infra/project-config master: tripleo: exclude reno jobs before Ocata  https://review.openstack.org/44760217:21
*** esikachev has quit IRC17:21
openstackgerritMohammed Naser proposed openstack-infra/infra-specs master: DockerHub Image Publisher  https://review.openstack.org/44752417:21
clarkbjeblair: ianw how are precise server replacement activities going? Anything I can do to help?17:21
*** isq has joined #openstack-infra17:21
mnaserso seems like most of kolla's folks are behind it so would love to start hearing from infra if anyone has free time ^17:22
openstackgerrityolanda.robla proposed openstack/diskimage-builder master: Apply setfiles on all mountpoints  https://review.openstack.org/44707617:22
*** skelso has quit IRC17:23
*** sekelso is now known as skelso17:24
clarkbmnaser: left one initial impression comment that may help further review17:25
mnaserthanks clarkb, i'll revise that once i have a few comments in (got a few $work things to finish up)17:26
clarkbsounds good17:26
*** rossella_s has quit IRC17:27
*** mat128 has quit IRC17:29
openstackgerrityolanda.robla proposed openstack/diskimage-builder master: Add dracut-regenerate elements  https://review.openstack.org/44704717:29
openstackgerritGage Hugo proposed openstack-dev/pbr master: Add Changelog build handling for invalid chars  https://review.openstack.org/43989717:30
clarkbsshnaidm: bkero EmilienM before I forget, I wanted to catch up on log collection things from late last week. Is there any other fs crawling I can do to help there or do we think its back under control again?17:31
*** rlandy is now known as rlandy|mtg17:31
sshnaidmclarkb, I think we have a couple of patches that should solve all finally17:32
sshnaidmclarkb, at least everything we discussed in bug/etherpad17:32
clarkbsshnaidm: ok not sure if you saw but the log collection was really good, then we regressed and shot back up again to over 300MB per job17:33
clarkbbkero had some changes in mind to address that, but not sure where they have gotten to17:33
sshnaidmclarkb, yeah, just reviewed his patch, I think it's this one: https://review.openstack.org/#/c/447174/17:33
*** mat128 has joined #openstack-infra17:34
*** nadya has quit IRC17:34
*** eroux has quit IRC17:34
*** mat128 has quit IRC17:34
*** jaosorior has quit IRC17:35
*** annegentle has joined #openstack-infra17:36
openstackgerritCorey O'Brien proposed openstack/diskimage-builder master: Ignore missing path in unmount_dir  https://review.openstack.org/44662117:39
clarkbsshnaidm: there were a bunch of container logs being grabbed too which incldued all of /etc for the containers17:40
clarkbso was like 5MB * num_containers17:40
*** baoli has quit IRC17:40
clarkband we also collected undercloud/ and 127.0.0.2/ logs which were duplicates17:40
sshnaidmclarkb, 127.0.0.2 is a bug, I think it's resolved17:41
*** mat128 has joined #openstack-infra17:41
sshnaidmclarkb, should be so: https://github.com/openstack-infra/tripleo-ci/blob/master/toci_instack_oooq.sh#L85-L8517:42
*** chlong has quit IRC17:42
*** baoli has joined #openstack-infra17:42
*** baoli has quit IRC17:42
*** masber has quit IRC17:43
sshnaidmclarkb, about container there is another his patch: https://review.openstack.org/#/c/447176/17:44
clarkbya just found that one. I see 01:05:14      bkero | and yet from TFPB:   hosts: all:!localhost re the 127.0.0.2 thing17:44
clarkbguessing thats a different job name?17:44
*** beslemon has quit IRC17:44
*** annegentle has quit IRC17:45
*** baoli has joined #openstack-infra17:46
*** masber has joined #openstack-infra17:46
clarkbsshnaidm: http://logs.openstack.org/57/446957/1/check-tripleo/gate-tripleo-ci-centos-7-ovb-containers-oooq-nv/bfb0438/logs/oooq/ is an example of 127.0.0.2 but not sure if more recent jobs are fixed17:47
*** neiljerram has quit IRC17:47
*** beslemon has joined #openstack-infra17:47
*** annegentle has joined #openstack-infra17:48
*** mat128 has quit IRC17:48
*** florianf has quit IRC17:50
*** nadya has joined #openstack-infra17:50
*** beslemon has quit IRC17:52
openstackgerritMerged openstack-infra/nodepool feature/zuulv3: Revert "Refactor nodepool apps into base app"  https://review.openstack.org/44753417:52
*** jpich has quit IRC17:54
*** jcoufal_ has joined #openstack-infra18:00
jeblairclarkb: i have not made the snapshot yet18:01
*** derekh has quit IRC18:02
*** lucasagomes is now known as lucas-afk18:02
*** ralonsoh has quit IRC18:02
*** jcoufal has quit IRC18:03
openstackgerritManjeet Singh Bhatia proposed openstack-infra/project-config master: Forcing tempest concurrency  https://review.openstack.org/44762818:05
*** masber has quit IRC18:05
*** mat128 has joined #openstack-infra18:06
*** aarefiev is now known as aarefiev_pto18:06
openstackgerritChandan Kumar proposed openstack-infra/tripleo-ci master: [TEST JOB] test tempest-12.2.0 for Mitaka  https://review.openstack.org/44686818:07
clarkbjeblair: should I go ahead and kick a snapshot off?18:07
*** nadya has quit IRC18:08
fungiclarkb: yeah, go ahead and start one. be prepared for it to take most of the day to complete since it's a conversion from a legacy instance (and so has a ~160gb rootfs)18:10
*** baoli has quit IRC18:10
*** mtanino has joined #openstack-infra18:11
clarkbnice18:11
*** e0ne has quit IRC18:11
clarkbok /me hops on puppetmaster to get that started18:11
*** mat128 has quit IRC18:11
fungibetter than the old wiki server, with its ~320gb rootfs18:11
fungiso maybe only a few hours18:11
*** baoli has joined #openstack-infra18:11
*** dtantsur is now known as dtantsur|afk18:12
*** nadya has joined #openstack-infra18:13
openstackgerritJames E. Blair proposed openstack-infra/nodepool feature/zuulv3: Break test_node_assignment_at_quota  https://review.openstack.org/44763018:13
jeblairclarkb: cool, thanks.  can you start an etherpad with info, so we can keep up with this semi-asynchronously?18:14
clarkbsure18:15
clarkbstep 0. determine osc command for making a snapshot18:15
clarkbjeblair: https://etherpad.openstack.org/p/lists.o.o-trusty-upgrade18:15
*** kzaitsev_mb has joined #openstack-infra18:16
*** baoli has quit IRC18:16
clarkbsnapshot create appears to be for volumes18:16
jeblairclarkb: we are both purple!18:17
openstackgerritSagi Shnaidman proposed openstack-infra/tripleo-ci master: Populate logs exclude list with infra requirements  https://review.openstack.org/44637418:17
clarkbjeblair: now I am green :)18:17
*** eroux has joined #openstack-infra18:17
*** adrian_otto has joined #openstack-infra18:17
*** annegentle has quit IRC18:17
*** armax has joined #openstack-infra18:18
*** tosky has quit IRC18:18
clarkbhttps://docs.openstack.org/user-guide/cli-use-snapshots-to-migrate-instances.html says you shutdown the server first?18:18
clarkb(I don't think you can actually do that in rax so guessing thats an optimization for making snapshot go faster)18:19
fungiwell, faster and ensures the fs is quiesced18:20
clarkbfungi: jeblair maybe you want to quiickly check the command on the etherpad before I run it? just to make sure I'm not doing something that will impact the running service18:21
fungibut if you don't mind maybe neeting to fsck the rootfs when booting from the snapshot, and the longer wait, it's fine18:21
fungis/neeting/needing/18:21
fungiclarkb: lgtm. we have active backups on that server too, right?18:22
fungiwhen we boot from that snapshot, need to make sure we disable the cronjob for backups so it doesn't fight with the real lists.o.o (same issue encountered with wiki.o.o)18:23
*** jamielennox|away is now known as jamielennox18:24
*** mat128 has joined #openstack-infra18:24
clarkbfungi: the bup objects dir for lists.o.o is timestamps march 20 at 1000UTC ish18:24
clarkbso yes I think we do have active backups still running there18:24
clarkboh good point18:25
funginoted in the etherpad now18:25
clarkbok starting the snapshot command on etherpad now18:25
fungithe good dale is in the lodge, and he can't leave. write it in your diary18:26
jeblairclarkb, fungi: it might be good to do it with mailman idle and empty queues... otherwise, when we boot it, mailman/exim might send out queued messages18:26
*** dharinic is now known as dharinic|lunch18:26
jeblairotoh, it's probably pretty hard for us to get empty queues18:26
*** rbrndt has quit IRC18:27
jeblairi wonder if we could boot it in rescue mode the first time?18:27
clarkbor maybe without networking (or does rescue imply that?)18:27
openstackgerritDavanum Srinivas (dims) proposed openstack-infra/project-config master: End Project Gating for python-k8sclient  https://review.openstack.org/44763518:28
clarkbsnapshot is running now fwiw18:28
*** baoli has joined #openstack-infra18:28
fungigood point, and i don't know if you can do a rescue boot unless you can tell it to create the server from the snapshot without booting it18:28
clarkbjeblair: fungi worst case you can probably doctor the image locally though downloading 160GB may take a while18:28
*** mat128 has quit IRC18:29
fungiwe could edit some configuration to purposefully break delivery (be that exim or iptables or dns or...) without actually reloading that configuration in production18:29
clarkbbut once its set to have the crontab cleaned up and mail queues emptied push back to rax and boot there for real world testing of uopgrade18:29
fungithen when it's read from the snapshot at test server boot time, it configures services such that delivery won't occur18:29
*** vsaienko has quit IRC18:29
fungii don't relish the idea of transferring 160gb images in and out of rackspace (not even sure uploading an image that large to glance will work there--we've never tried it to my knowledge)18:30
*** vhosakot has joined #openstack-infra18:30
*** mat128 has joined #openstack-infra18:31
jeblairyeah, next time we can probably disable the init scripts18:31
fungioh, yep18:31
jeblairis there a way to abort the snapshot so we can try again?18:31
clarkbI am not aware of one :/18:32
jeblairor should we just try again tomorrow?  or should we proceed with this and just try to be really quick and hope not much mail sneaks out?18:32
*** baoli has quit IRC18:32
fungithe "delete image" action for it in the webui isn't greyed out18:33
fungiif that's something we want to try18:33
fungiwe can't create server or share the image yet, but it seems willing to let us rename or delete even though it's in a saving state18:34
clarkbassuming this only takes a few hours we could try again after lunch18:34
clarkbfungi: I'd worry about the image "leaking" but maybe thats not a huge deal18:34
jeblairok, let's see where we are then18:34
*** sree has joined #openstack-infra18:34
fungiwfm18:35
*** nadya has quit IRC18:35
AJaegerclarkb, https://review.openstack.org/417555 is merged - the py3 devstack change -  but having two tox environments run breaks the freeze command, see http://logs.openstack.org/95/447195/2/check/gate-devstack-gate-unittests/0ba2a39/console.html#_2017-03-18_09_16_38_288112 . Anybody an idea how to move forward here? I see two options: 2 separate tox jobs or adjusting run-tox.sh script.18:36
*** ssbarnea has quit IRC18:38
*** baoli has joined #openstack-infra18:38
clarkbAJaeger: we can run 2 separate tox jobs, that is probably a better way to organize it givne how we do other jobs18:38
*** ociuhandu has quit IRC18:38
*** sree has quit IRC18:39
AJaegerclarkb: do you have time to do a change or shall I?18:39
clarkbAJaeger: I can do it, just have to page that work back in18:39
openstackgerrityolanda.robla proposed openstack/diskimage-builder master: Apply setfiles on all mountpoints  https://review.openstack.org/44707618:39
clarkbAJaeger: this is project-config only ya?18:40
AJaegerclarkb: yes18:40
*** gus has quit IRC18:41
openstackgerrityolanda.robla proposed openstack/diskimage-builder master: Apply setfiles on all mountpoints  https://review.openstack.org/44707618:42
AJaegerclarkb: I wonder why gate-devstack-gate-unittests is a separate job, we could use the default tox template for this, can't we?18:42
openstackgerritClark Boylan proposed openstack-infra/project-config master: Run separate jobs for py2 and py3 d-g unittests  https://review.openstack.org/44764118:43
clarkbAJaeger: thats a good question actually18:43
*** baoli has quit IRC18:44
*** xarses_ has joined #openstack-infra18:44
AJaegerone difference: the current test does not use the test-results publisher18:45
clarkbit also uses gerrit git prep. but otherwise I think 'gate-{name}-tox-{envlist}-{node}' should be a safe substitute18:45
*** gus has joined #openstack-infra18:45
clarkbwould you like me to switch to that instead?18:45
AJaegerclarkb: Yes, I think that's a nice cleanup we should do - but I would not -1 your on that.18:46
clarkbsshnaidm: bkero http://logs.openstack.org/97/442497/2/check-tripleo/gate-tripleo-ci-centos-7-ovb-containers-oooq-nv/db3ba3d/logs/oooq/ is a run from just now and includes 127.0.0.2/ logs18:47
clarkbAJaeger: ok maybe get my current change in quickly now since its small and fixes a problem and I will followup with a change that changes the builder18:47
AJaegerclarkb: ok18:48
* AJaeger reviews18:48
*** andreww has quit IRC18:48
*** andreww has joined #openstack-infra18:48
*** xarses_ has quit IRC18:49
*** rlandy|mtg is now known as rlandy18:49
EmilienMproject-config cores: I have 3 patches ready for review (thanks!) : https://review.openstack.org/#/q/project:openstack-infra/project-config+owner:%22Emilien+Macchi+%253Cemilien%2540redhat.com%253E%22+status:open18:50
openstackgerritClark Boylan proposed openstack-infra/project-config master: Use standard tox builder for d-g unittests  https://review.openstack.org/44764518:51
clarkbAJaeger: ^ there is followup18:51
openstackgerritDavanum Srinivas (dims) proposed openstack-infra/project-config master: End Project Gating for python-k8sclient  https://review.openstack.org/44763518:51
*** david-lyle_ has joined #openstack-infra18:52
*** david-lyle has quit IRC18:52
*** david-lyle_ is now known as david-lyle18:53
AJaegerthanks, clarkb18:54
AJaegerEmilienM: +418:54
*** ym200 has quit IRC18:54
*** eharney has quit IRC18:55
sshnaidmclarkb, that's very strange, will look..18:55
*** iOx has joined #openstack-infra18:57
EmilienMAJaeger: thanks18:57
openstackgerritJesse Keating proposed openstack-infra/zuul feature/zuulv3: support github pull reqeust labels  https://review.openstack.org/44451118:58
openstackgerritJesse Keating proposed openstack-infra/zuul feature/zuulv3: Allow github trigger to match on branches/refs  https://review.openstack.org/44562518:58
openstackgerritJesse Keating proposed openstack-infra/zuul feature/zuulv3: Set filter according to PR/Change in URL  https://review.openstack.org/44678218:58
openstackgerritJesse Keating proposed openstack-infra/zuul feature/zuulv3: Support for dependent pipelines with github  https://review.openstack.org/44529218:58
openstackgerritJesse Keating proposed openstack-infra/zuul feature/zuulv3: Add basic Github Zuul Reporter.  https://review.openstack.org/44332318:58
openstackgerritJesse Keating proposed openstack-infra/zuul feature/zuulv3: Configurable SSH access to GitHub  https://review.openstack.org/44403418:58
openstackgerritJesse Keating proposed openstack-infra/zuul feature/zuulv3: Add 'push' and 'tag' github webhook events.  https://review.openstack.org/44394718:58
openstackgerritJesse Keating proposed openstack-infra/zuul feature/zuulv3: Add 'pr-comment' github webhook event  https://review.openstack.org/44395918:58
openstackgerritJesse Keating proposed openstack-infra/zuul feature/zuulv3: Support for github commit status  https://review.openstack.org/44406018:58
openstackgerritJesse Keating proposed openstack-infra/zuul feature/zuulv3: Better merge message for GitHub pull reqeusts  https://review.openstack.org/44564418:58
openstackgerritJesse Keating proposed openstack-infra/zuul feature/zuulv3: Encapsulate determining the event purpose  https://review.openstack.org/44524218:58
openstackgerritJesse Keating proposed openstack-infra/zuul feature/zuulv3: Allow using webapp from connections  https://review.openstack.org/43983118:58
openstackgerritJesse Keating proposed openstack-infra/zuul feature/zuulv3: Merge pull requests from github reporter  https://review.openstack.org/44446318:58
openstackgerritJesse Keating proposed openstack-infra/zuul feature/zuulv3: Support GitHub PR webhooks  https://review.openstack.org/43983418:58
openstackgerritJesse Keating proposed openstack-infra/zuul feature/zuulv3: GitHub file matching support  https://review.openstack.org/44611318:58
openstackgerritJesse Keating proposed openstack-infra/zuul feature/zuulv3: Log GitHub API rate limit  https://review.openstack.org/44615018:58
*** masber has joined #openstack-infra18:58
*** beslemon has joined #openstack-infra18:59
*** dizquierdo has joined #openstack-infra19:00
*** ym200 has joined #openstack-infra19:00
openstackgerritEmilien Macchi proposed openstack-infra/project-config master: tripleo: exclude reno jobs before Ocata  https://review.openstack.org/44760219:00
EmilienMAJaeger: alphabeticaly ordered :D ^19:00
*** armax has quit IRC19:01
*** iOx has quit IRC19:01
openstackgerritPaul Belanger proposed openstack-infra/nodepool feature/zuulv3: Remove legacy openstack settings from nodepool.yaml  https://review.openstack.org/44764719:02
*** sekelso has joined #openstack-infra19:03
*** kzaitsev_mb has quit IRC19:04
*** skelso has quit IRC19:05
*** baoli has joined #openstack-infra19:07
AJaegerEmilienM: nice ;)19:07
openstackgerritSagi Shnaidman proposed openstack-infra/tripleo-ci master: Use separate log collection playbook for TripleO CI  https://review.openstack.org/44764819:08
openstackgerritSagi Shnaidman proposed openstack-infra/tripleo-ci master: Use separate log collection playbook for TripleO CI  https://review.openstack.org/44764819:09
openstackgerritMerged openstack-infra/jenkins-job-builder master: Add support for 'Groovy Label Assignment Plugin'  https://review.openstack.org/42841919:10
openstackgerritDavid Shrewsbury proposed openstack-infra/nodepool feature/zuulv3: Fix test_node_assignment_at_quota  https://review.openstack.org/44763019:10
*** eharney has joined #openstack-infra19:11
openstackgerritDavid Shrewsbury proposed openstack-infra/nodepool feature/zuulv3: Fix test_node_assignment_at_quota  https://review.openstack.org/44763019:11
*** dharinic|lunch is now known as dharinic19:13
*** baoli has quit IRC19:14
*** tosky has joined #openstack-infra19:14
*** masber has quit IRC19:17
*** adrianofr_ has quit IRC19:19
*** gyee has joined #openstack-infra19:20
*** hasharAway is now known as hashar19:22
*** baoli has joined #openstack-infra19:25
*** vsaienko has joined #openstack-infra19:26
openstackgerritwes hayutin proposed openstack-infra/tripleo-ci master: WIP: create a README_LOGS to document log files  https://review.openstack.org/44765319:28
*** _ari_|gone is now known as _ari_19:31
*** adrian_otto has quit IRC19:34
*** rbrndt has joined #openstack-infra19:40
openstackgerrityolanda.robla proposed openstack/diskimage-builder master: Apply setfiles on all mountpoints  https://review.openstack.org/44707619:41
*** esikache1 has quit IRC19:42
*** krtaylor has quit IRC19:43
*** beslemon_ has joined #openstack-infra19:45
*** xarses_ has joined #openstack-infra19:46
openstackgerritPaul Belanger proposed openstack-infra/nodepool feature/zuulv3: Remove legacy openstack settings from nodepool.yaml  https://review.openstack.org/44764719:47
*** beslemon has quit IRC19:47
fungiSotK: Zara: I'm getting ready to do the sb migrations for monasca, refstack and defcore if we're still all go for that19:47
fungi12 minutes by my watch19:48
fungihappen to know if we had open project-config changes for those?19:48
*** andreww has quit IRC19:48
*** masber has joined #openstack-infra19:50
*** rockyg has joined #openstack-infra19:53
*** matt-borland has quit IRC19:54
*** mat128 has quit IRC19:54
*** mozart89 has joined #openstack-infra19:56
*** e0ne has joined #openstack-infra19:58
* SotK doesn't know if we do20:00
fungii assumed we don't and am almost done writing20:00
*** baoli has quit IRC20:01
*** baoli has joined #openstack-infra20:01
* SotK notices that https://review.openstack.org/#/c/444595/ didn't get merged20:01
*** baoli has quit IRC20:02
fungiahh, skimming that change, we should probably just use the pip package provider for those rather than explicitly invoking pip with an exec. among other reasons this will rerun that command every time puppet runs20:03
fungii'll push up a new patchset for it in just a moment20:04
openstackgerritMerged openstack-infra/zuul feature/zuulv3: Fix testtools.run syntax for README.rst  https://review.openstack.org/44721920:04
*** dprince has quit IRC20:05
openstackgerritJeremy Stanley proposed openstack-infra/project-config master: Use StoryBoard for Monasca repositories  https://review.openstack.org/44767220:06
*** rockyg has quit IRC20:06
*** rockyg has joined #openstack-infra20:06
SotKah, TIL about that, thanks20:06
*** eroux has quit IRC20:07
clarkbjeblair: fungi image show on the lists snapshot shows its active now20:07
clarkbso that only took a couple hours I think20:07
fungiclarkb: oh, much faster than i anticipated20:07
*** nadya has joined #openstack-infra20:09
openstackgerritPaul Belanger proposed openstack-infra/nodepool feature/zuulv3: Remove keypair from provider section  https://review.openstack.org/44767320:09
jeblairclarkb: should we make another one now after disabling boot scripts?20:10
clarkbjeblair: sure I think that may make booting later simpler20:10
clarkbyou might want to disable puppet on the host ebfore hand too20:11
clarkbas its possible puppet could undo these temporary actions during the snapshot20:11
SotKfungi: I think currently all monasca bugs are against a single "monasca" project in LP, what is the intended destination project in SB going to be?20:11
*** esikache1 has joined #openstack-infra20:11
*** baoli has joined #openstack-infra20:12
fungiSotK: those all end up in the monasca project in lp too20:12
*** nadya has quit IRC20:12
*** matrohon has joined #openstack-infra20:12
fungiwitek: you're i guess the closest thing we have to a monasca liaison for infra (doesn't look like rhochmuth is around, even in #openstack-monasca)... looking to get some last-minute feedback on https://review.openstack.org/447672 before starting the maintenance (to make sure i have the correct repos flagged)20:13
*** masber has quit IRC20:13
jeblairclarkb: i'll disable puppet20:14
*** dave-mcc_ has joined #openstack-infra20:15
SotKfungi: s/lp/sb/?20:15
*** jiaohaolin2 has joined #openstack-infra20:15
*** rakhmerov has quit IRC20:15
*** rakhmerov__ has joined #openstack-infra20:15
*** kencjohnston_ has joined #openstack-infra20:15
*** abramley_ has joined #openstack-infra20:15
*** knangia_ has joined #openstack-infra20:15
fungiSotK: currently lp until we migrate them20:16
*** Wei_Liu1 has joined #openstack-infra20:16
*** esikache1 has quit IRC20:16
*** pbourke_ has joined #openstack-infra20:16
jeblairclarkb, fungi: i ran update-rc.d mailman disable20:17
jeblairalso exim420:17
*** bramwelt1 has joined #openstack-infra20:17
*** Sukhdev_ has joined #openstack-infra20:17
jeblairclarkb, fungi: i think we can kick off another snapshot now?20:17
jeblairoh wait20:17
*** adrian_otto has joined #openstack-infra20:17
jeblairi'll disable bup too20:17
fungiSotK: that sentence in the commit message is pointing out that monasca-events-api has been using the monasca project in lp, so it probably just overlooked for addition to governance as an official deliverable repo20:17
*** zhenguo_ has joined #openstack-infra20:18
clarkbjeblair: cool let me know if you want me to rerun the snapshot or if you will do it20:18
*** pleia2_ has joined #openstack-infra20:18
jeblairclarkb: bup is disabled;  why don't you rerun the snapshot now20:18
fungijeblair: i guess it's okay if puppet tries to reenable those while the snapshot is being generated?20:18
*** Vivek__ has joined #openstack-infra20:18
*** mdrabe_ has joined #openstack-infra20:18
fungior did you also disable lists.o.o in puppet?20:18
jeblairfungi: i disabled puppet at clarkb's suggestion20:18
fungiaha, i missed that--thanks20:18
*** lucas-afk has quit IRC20:18
*** ijw has quit IRC20:18
clarkbjeblair: rerunning, anmed it precise-lists.o.o-20170320.220:19
*** makowals_ has joined #openstack-infra20:19
*** daniel1 has joined #openstack-infra20:19
*** kgiusti has quit IRC20:19
SotKfungi: alright, will that patch cause a single monasca project to be created in SB then rather than one-per-repo like we have for most (all?) other projects currently in there?20:20
*** jamesdenton has quit IRC20:20
*** rook has quit IRC20:20
*** vsaienk01 has joined #openstack-infra20:20
*** wfoster has quit IRC20:20
fungiSotK: it will create individual sb "projects" for each repo but put them each in a group called "monasca"20:20
*** rosmaita_ has joined #openstack-infra20:20
*** tinwood_ has joined #openstack-infra20:21
*** rook has joined #openstack-infra20:21
fungithat's how we did all the infra repos anyway20:21
*** krtaylor has joined #openstack-infra20:21
*** lucasagomes has joined #openstack-infra20:21
*** BobBall_ has joined #openstack-infra20:21
*** rook is now known as Guest4502520:21
*** sekelso has quit IRC20:21
*** sekelso has joined #openstack-infra20:22
fungiand the reason why we renamed that option in jeepyb to "groups" instead of launchpad-group (implying that it's not lp-specific and could include more than one, at least in an sb context)20:22
*** jmccrory_ has joined #openstack-infra20:22
*** toabctl_ has joined #openstack-infra20:22
*** DuncanT_ has joined #openstack-infra20:22
*** sheeprine_ has joined #openstack-infra20:22
*** e0ne has quit IRC20:22
*** amrith has quit IRC20:23
*** GregHous- has joined #openstack-infra20:23
*** amrith has joined #openstack-infra20:23
*** baoli has quit IRC20:23
*** wfoster has joined #openstack-infra20:23
*** Gorian- has joined #openstack-infra20:23
SotKthe migration script currently only supports importing from one LP project into one SB project, so I guess there may need to be a manual sorting step for someone to assign the created tasks to the correct projects after the fact20:23
*** lunarlamp has joined #openstack-infra20:24
*** sdake_ has joined #openstack-infra20:24
*** sdake_ has quit IRC20:24
*** sdake_ has joined #openstack-infra20:24
fungiSotK: right, the assumption is that they'll all show up as stories for openstack/monasca (at least initially)20:24
*** puiterwi1 has joined #openstack-infra20:24
zigopabelanger: Any news from puppetlabs about supporting Stretch?20:24
SotKalright then, I am no longer concerned :)20:25
*** radez` has joined #openstack-infra20:25
openstackgerritWitold Bedyk proposed openstack-infra/project-config master: Add {name}-branch-tarball for Monasca projects  https://review.openstack.org/44753920:25
openstackgerritJeremy Stanley proposed openstack-infra/project-config master: Use StoryBoard for Interop repositories  https://review.openstack.org/44767520:25
fungiSotK: ^ there's the interop one20:26
*** jcoufal has joined #openstack-infra20:26
*** armax has joined #openstack-infra20:26
pabelangerzigo: nothing yet, but I looked at it over the weekend. should be fine to proceed, since we use OS puppet for xenial20:26
*** xhku_ has joined #openstack-infra20:26
openstackgerritPaul Belanger proposed openstack-infra/nodepool feature/zuulv3: Remove keypair from provider section  https://review.openstack.org/44767320:27
*** burgerk has quit IRC20:27
openstackgerritPaul Belanger proposed openstack-infra/nodepool feature/zuulv3: Remove legacy openstack settings from nodepool.yaml  https://review.openstack.org/44764720:27
openstackgerritPaul Belanger proposed openstack-infra/nodepool feature/zuulv3: Remove SSH support from nodepool  https://review.openstack.org/44668320:27
*** askb has joined #openstack-infra20:27
*** Guest45025 is now known as rook-ugh20:27
*** jgriffith_ has joined #openstack-infra20:27
openstackgerritTin Lam proposed openstack-infra/project-config master: Add releasenotes job for kuryr  https://review.openstack.org/44067620:27
openstackgerritPaul Belanger proposed openstack-infra/nodepool feature/zuulv3: Remove keypair from provider section  https://review.openstack.org/44767320:28
openstackgerritPaul Belanger proposed openstack-infra/nodepool feature/zuulv3: Remove legacy openstack settings from nodepool.yaml  https://review.openstack.org/44764720:28
*** jcoufal_ has quit IRC20:29
*** jkilpatr has quit IRC20:29
*** knangia has quit IRC20:29
*** mdrabe has quit IRC20:29
*** zul has quit IRC20:29
*** _Cyclone_ has quit IRC20:29
*** dave-mccowan has quit IRC20:29
*** AJaeger has quit IRC20:29
*** pbourke has quit IRC20:29
*** makowals has quit IRC20:29
*** jiaohaolin1 has quit IRC20:29
*** Wei_Liu has quit IRC20:29
*** quite has quit IRC20:29
*** GregHouse has quit IRC20:29
*** tinwood has quit IRC20:29
*** sdague has quit IRC20:29
*** sheeprine has quit IRC20:29
*** soliosg has quit IRC20:29
*** vsaienk0 has quit IRC20:29
*** DuncanT has quit IRC20:29
*** Odd_Bloke has quit IRC20:29
*** BobBall has quit IRC20:29
*** sdake has quit IRC20:29
*** Gorian has quit IRC20:29
*** Vivek has quit IRC20:29
*** zhenguo has quit IRC20:29
*** bramwelt has quit IRC20:29
*** jmccrory has quit IRC20:29
*** jgriffith has quit IRC20:29
*** toabctl has quit IRC20:29
*** fbouliane has quit IRC20:29
*** kencjohnston has quit IRC20:29
*** puiterwijk has quit IRC20:29
*** abramley has quit IRC20:29
*** pleia2 has quit IRC20:29
*** mariusv has quit IRC20:29
*** radez has quit IRC20:29
*** rosmaita has quit IRC20:29
*** soliosg1 is now known as soliosg20:29
*** jmccrory_ is now known as jmccrory20:29
*** abramley_ is now known as abramley20:29
*** lunarlamp is now known as mariusv20:29
*** toabctl_ is now known as toabctl20:29
*** Wei_Liu1 is now known as Wei_Liu20:29
*** sheeprine_ is now known as sheeprine20:29
*** sheeprine has quit IRC20:29
*** sheeprine has joined #openstack-infra20:29
*** sheeprine has quit IRC20:29
*** sheeprine has joined #openstack-infra20:29
*** knangia_ is now known as knangia20:29
*** zhenguo_ is now known as zhenguo20:30
*** jgriffith_ is now known as jgriffith20:30
*** AJaeger has joined #openstack-infra20:31
*** zul has joined #openstack-infra20:31
fungiSotK: even better, there are distro packages of both of those in trusty already so we can just use those instead20:31
*** quite has joined #openstack-infra20:32
*** shardy has quit IRC20:32
*** DuncanT_ is now known as DuncanT20:32
*** jkilpatr has joined #openstack-infra20:33
*** dave-mcc_ is now known as dave-mccowan20:33
SotKfungi: excellent20:33
openstackgerritJeremy Stanley proposed openstack-infra/puppet-storyboard master: Install Launchpad migration script dependencies  https://review.openstack.org/44459520:33
fungiSotK: ^ there you go20:34
ianwclarkb: was going to spin that up today, in about 2 hours.  sorry got a little sidetracked with dib stuff20:34
clarkbianw: no problem just wanted to make suer it was still on peoples radars. Let me know if I can help out20:35
SotKfungi: thanks20:35
*** sdague has joined #openstack-infra20:36
*** sdague has quit IRC20:37
*** baoli has joined #openstack-infra20:37
openstackgerritJeremy Stanley proposed openstack-infra/project-config master: Use StoryBoard for Interop repositories  https://review.openstack.org/44767520:37
*** soliosg1 has joined #openstack-infra20:37
fungihogepodge: ^ i'm trying to strike a balance there between the refstack team and the interop working group20:38
funginot sure if i've got the distinction quite right, but if you have a moment to skim it20:38
openstackgerritPaul Belanger proposed openstack-infra/nodepool feature/zuulv3: Remove keypair from provider section  https://review.openstack.org/44767320:38
openstackgerritPaul Belanger proposed openstack-infra/nodepool feature/zuulv3: Remove legacy openstack settings from nodepool.yaml  https://review.openstack.org/44764720:38
*** beslemon_ has quit IRC20:38
openstackgerritMerged openstack-infra/nodepool feature/zuulv3: Unlock request if it disappears  https://review.openstack.org/44750120:39
*** shardy has joined #openstack-infra20:39
openstackgerritGage Hugo proposed openstack-dev/pbr master: Add Changelog build handling for invalid chars  https://review.openstack.org/43989720:39
asselin_Hi, when setting up a new gerrit instance and using jeepyb, how does the jeepyb's user name and e-mail get configured (to match the openstack-project-creator user's e-mail) ?20:39
*** jmorgan1 has quit IRC20:39
asselin_I'm getting this error: ! [remote rejected]     HEAD -> refs/meta/config (invalid committer)20:40
*** Gorian- is now known as Gorian20:41
fungiasselin_: jeepyb has never solved the initial bootstrapping problem20:41
fungiasselin_: so you need to manually set that part up20:42
asselin_fungi: ok...so just manually run the git config --global user.name etc. commands?20:42
asselin_ack...thanks for confirming20:42
clarkbI think our puppet may do it for us, but ya jeepyb won't itself20:42
fungiasselin_: oh, that part of the bootstrapping. it's possible puppet-gerrit tries to take care of that, but maybe not?20:43
fungiasselin_: are you using the puppet module or some other installation mechanism?20:43
asselin_well I see the desired e-amil the gerrit config file...yes using the puppet module...I dig deeper there...maybe something didn't work with that20:43
*** adriant has joined #openstack-infra20:44
*** sdague has joined #openstack-infra20:45
*** jmorgan1 has joined #openstack-infra20:46
*** ZZelle has joined #openstack-infra20:46
openstackgerritPaul Belanger proposed openstack-infra/zuul feature/zuulv3: [WIP] Fix zuul-nodepool-integration job  https://review.openstack.org/44768320:48
fungiasselin_: some of our persistent job nodes (release.slave.o.o, signing01.ci.o.o, proposal.slave.o.o) push changes or tags into gerrit and so do some dance in puppet to set up git configuration for the appropriate account (i think), so if puppet-gerrit is missing that bit you could presumably crib from one of them20:50
clarkbthe jenkins jobs actually do it in job iirc20:50
clarkbsince its an easy low overhead way to make sure its correct for each run20:51
*** sdague has quit IRC20:51
*** sdague__ has joined #openstack-infra20:52
*** dizquierdo has quit IRC20:55
*** ijw has joined #openstack-infra20:55
openstackgerritTin Lam proposed openstack-infra/project-config master: Add releasenotes job for kuryr  https://review.openstack.org/44067620:55
*** ijw has quit IRC20:56
*** ijw has joined #openstack-infra20:56
openstackgerritKaitlin Farr proposed openstack-infra/project-config master: Add Barbican grenade gate  https://review.openstack.org/44768920:57
ihrachysclarkb: maybe revisit https://review.openstack.org/#/c/383827/ ? I think you +2d it before but now it's a tad different (per your suggestion)20:57
*** puiterwi1 is now known as puiterwijk20:59
*** beslemon has joined #openstack-infra20:59
clarkbihrachys: looking at https://review.openstack.org/#/c/383827/17/devstack-vm-gate.sh do we need to set it to dvr_snat if all in one (aio)?21:00
fungiclarkb: oh, yeah i think we had some of that in puppet for the proposal node originally or something? but could be all in-job now (except for installing the ssh keys)21:00
*** rhallisey has quit IRC21:00
clarkbihrachys: I think we set it to dvr_snat on all in one with old code but new code would set it to dvr21:00
zigopabelanger: Xenial uses puppet 3, not 4 like in Stretch.21:00
zigopabelanger: So there may be some fixes to apply...21:00
clarkbihrachys: but I am not sure what all in one requires there if anything21:00
ihrachysclarkb: oh, yeah, I think you are right, because otherwise it would not do SNAT for aio21:01
zigopabelanger: Anyway, my point of view is always to fixup upward compat issues rather than adding band-aid patches to use lower versions.21:01
ihrachyswhich is a weird setup for DVR but still21:01
clarkbihrachys: so we just need a third condition of elif aio then dvr_snat else dvr21:01
zigopabelanger: Any idea how to fix the "import snmptd" that fails then?21:01
clarkbI will elave a comment21:02
*** sekelso has quit IRC21:05
*** sekelso has joined #openstack-infra21:05
fungican i get an infra-puppet-core review on 444595 since i need these libs installed on the storyboard server for running the lp migration script?21:05
*** jiaohaolin2 has quit IRC21:05
* clarkb lokos21:06
fungivery tiny patch ;)21:06
*** jiaohaolin1 has joined #openstack-infra21:06
*** dimtruck is now known as zz_dimtruck21:07
clarkbfungi: and you are fine with both of those coming from distro and not pip?21:07
fungiclarkb: yeah, they're not declared deps of sb or else pip would be dragging them in when it installs the git checkout21:08
*** baoli has quit IRC21:09
fungiin the past we've preferred distro packages over random pip installs into system context when we can get away with it21:09
clarkbya just wasn't sure if the versions were new enough etc21:09
fungiwell, i'll know when i get started. i could preinstall them and test instead of letting puppet do it, but figured i'd just get config management out of the way up front21:10
*** baoli has joined #openstack-infra21:11
*** baoli has quit IRC21:11
*** baoli has joined #openstack-infra21:11
*** eharney has quit IRC21:12
*** Goneri has quit IRC21:12
fungithanks clarkb21:13
*** sekelso has quit IRC21:14
*** mdrabe_ is now known as mdrabe21:14
*** ihrachys has quit IRC21:14
*** apetrich has quit IRC21:14
pabelangerzigo: no, I'd need to see the build log again21:15
*** apetrich has joined #openstack-infra21:15
zigopabelanger: To me, something must be destroying the puppet's modulepath.21:17
*** ldnunes has quit IRC21:18
*** ym200 has quit IRC21:20
openstackgerritMerged openstack-infra/puppet-storyboard master: Install Launchpad migration script dependencies  https://review.openstack.org/44459521:21
*** ym200 has joined #openstack-infra21:21
*** EricGonc_ has quit IRC21:21
*** amotoki has joined #openstack-infra21:23
openstackgerritMonty Taylor proposed openstack-infra/shade master: Use data when the data has a non-json content type  https://review.openstack.org/44758121:24
*** baoli has quit IRC21:24
openstackgerritMerged openstack-infra/project-config master: Use StoryBoard for Monasca repositories  https://review.openstack.org/44767221:27
openstackgerritMonty Taylor proposed openstack-infra/shade master: Use data when the request has a non-json content type  https://review.openstack.org/44758121:27
*** aysyd has quit IRC21:29
mmedvedeianw: I'd llke to follow up on http://lists.openstack.org/pipermail/openstack-dev/2017-March/114263.html with you or with Andre21:30
fungican i get a project-config-core reviewer's eyes on 447675 so i can work on migrating them shortly?21:30
mmedvedeianw: I got a diskimage-builder test job working in third-party CI on power, can I enable it to report results to diskimage-builder patches?21:31
mmedvedeianw: and do you know what Andre's irc nick is?21:31
*** baoli has joined #openstack-infra21:32
*** baoli has quit IRC21:33
clarkbfungi: yup can do that now (was afk for a short bit but back now)21:34
clarkbfungi: is being in multiple groups valid?21:35
*** Jeffrey4l_ has quit IRC21:35
*** ihrachys has joined #openstack-infra21:35
*** ijw has quit IRC21:35
clarkbgrep says we do it for sahara21:36
fungiclarkb: it's supposed to be valid in sb21:36
*** shardy has quit IRC21:37
*** ihrachys has quit IRC21:37
openstackgerritMonty Taylor proposed openstack-infra/shade master: Get the ball rolling on magnumclient  https://review.openstack.org/44754321:37
openstackgerritMonty Taylor proposed openstack-infra/shade master: Add normalization for cluster templates  https://review.openstack.org/44754421:37
openstackgerritMonty Taylor proposed openstack-infra/shade master: RESTify cluster template tests  https://review.openstack.org/44754521:37
openstackgerritMonty Taylor proposed openstack-infra/shade master: Convert magnum service to requests_mock  https://review.openstack.org/44754621:37
openstackgerritMonty Taylor proposed openstack-infra/shade master: Get rid of magnumclient dependency  https://review.openstack.org/44757821:37
*** Jeffrey4l_ has joined #openstack-infra21:38
funginot sane in lp afaik since the behavior for that field in the face of lp bugs is to use the one and only groups list member as the lp project name for bugs, while for sb it actually indicates the sb groups of which a project should be a member21:38
fungiwe implemented it that way in jeepyb as a transitional option with different but related meanings for lp vs sb21:39
clarkbjeblair: precise-lists.o.o-20170320.2 is ready (active)21:40
fungii imagine he's not going to do much with it in the next 20 minutes leading up to the zuul meeting21:41
fungibut awesome21:41
fungiclarkb: have the ip address handy?21:41
* fungi is going to peek at the disabled things to see that they're still disabled in the clone21:42
clarkbfungi: just the snapshot, no booted instance21:42
*** zz_dimtruck is now known as dimtruck21:42
fungioh, right21:45
openstackgerritMerged openstack-infra/project-config master: Use StoryBoard for Interop repositories  https://review.openstack.org/44767521:47
*** matrohon has quit IRC21:49
*** xarses_ has quit IRC21:50
*** gouthamr has quit IRC21:50
*** mhickey has quit IRC21:53
*** vhosakot has quit IRC21:53
jheskethMorning21:54
*** mat128 has joined #openstack-infra21:55
bkerojeblair: when you have time: https://review.openstack.org/#/c/445660/21:55
*** daniel1 is now known as Odd_Bloke21:56
openstackgerritBen Kero proposed openstack-infra/tripleo-ci master: ovb: Remove bin/ and usr/ from working_dir tarball  https://review.openstack.org/44717421:57
*** xarses_ has joined #openstack-infra21:57
*** andreww has joined #openstack-infra21:58
*** ym200 has quit IRC21:59
*** ym200 has joined #openstack-infra22:00
jeblairit's zuul meeting time in #openstack-meeting-alt22:00
openstackgerritBen Kero proposed openstack-infra/tripleo-ci master: Revert second repo-setup run for overcloud nodes  https://review.openstack.org/44714222:00
*** mat128 has quit IRC22:01
*** hashar has quit IRC22:01
*** baoli has joined #openstack-infra22:01
*** xyang1 has quit IRC22:01
*** xarses_ has quit IRC22:02
*** jkilpatr has quit IRC22:03
*** baoli has quit IRC22:06
*** ijw has joined #openstack-infra22:06
*** sdake_ is now known as sdake22:07
*** lyarwood has quit IRC22:07
*** kaisers has quit IRC22:08
*** lyarwood has joined #openstack-infra22:09
*** rfolco has quit IRC22:10
*** dave-mccowan has quit IRC22:12
openstackgerritClint 'SpamapS' Byrum proposed openstack-infra/infra-specs master: Zuulv3 Executor Security Enhancement  https://review.openstack.org/44449522:14
fungizuul the executioner!22:15
SpamapSkneel, VM scum.22:16
*** mdrabe has quit IRC22:17
*** ym200 has quit IRC22:19
*** jcoufal has quit IRC22:19
openstackgerritMerged openstack-infra/system-config master: Remove extra sample ATCs file  https://review.openstack.org/44214822:20
asselin_clarkb: fungi FYI: seems to be an issue b/w the git author (which is being used correctly by jeepyb and matches what is in gerrit) and commiter e-mail which is currently the default user/e-mail setup by git but unknown to gerrit.22:20
fungiasselin_: oh, right, you can use the gerrit api to add that address to the specified account even if you didn't create the account with that address22:21
openstackgerritMerged openstack-infra/system-config master: Use PYTHONIOENCODING=utf-8 for invites invocation  https://review.openstack.org/44215822:21
fungiasselin_: oh, i misread22:21
fungiasselin_: so you just need .gitconfig to have the correct committer e-mail in it?22:22
openstackgerritMerged openstack-infra/system-config master: Eat the extra leading newline in invites  https://review.openstack.org/44215922:22
openstackgerritMerged openstack-infra/system-config master: Add human name to sample of invites from address  https://review.openstack.org/44216022:22
*** rockyg has quit IRC22:22
asselin_fungi: ok...I added the e-mail to gerrit's user and that worked. I'll see about the .gitconfig file...didn't knwo about it22:22
*** kaisers has joined #openstack-infra22:23
fungiasselin_: if adding the committer address into gerrit worked, then that's what was probably wrong and not the .gitconfig22:23
fungior put another way, you committer address needs to be known to gerrit, set on the account you've created and, if necessary, configured locally in git22:24
asselin_fungi: ok, then I'll submit a doc update patch with that note. I don't see any of this in .gitconfig nor git config -l22:24
fungiyeah, creating and configuring the project creator account is probably left as an exercise for the reader there22:25
asselin_actually quite a bit documented which was very helpful to me22:26
fungiand unless you're intimately familiar with deploying gerrit already, may not be easy to follow22:26
*** ym200 has joined #openstack-infra22:26
*** ijw has quit IRC22:26
*** neiljerram has joined #openstack-infra22:30
*** rook-ugh is now known as rook22:30
*** rook is now known as Guest838322:31
openstackgerritMerged openstack-infra/shade master: change test_endpoints to use requests mock  https://review.openstack.org/44454122:33
*** mriedem1 has joined #openstack-infra22:36
ianwmmedvede: looking ...22:36
*** mriedem has quit IRC22:38
ianwmmedvede: ahh, the ppc stuff, ok cool22:38
ianwmmedvede: andreas-f in #openstack-dib, we can discuss there or i'll reply in a bit22:39
*** gordc has quit IRC22:40
openstackgerritPaul Belanger proposed openstack-infra/zuul feature/zuulv3: Add ansible-lint to tox (pep8)  https://review.openstack.org/43827622:40
fungiSotK: if you're still around, looks like jeepyb successfully created the projects and groups on sb.o.o and puppet installed our migrate deps... looking at the output of storyboard-migrate --help there are rather a lot of options. do you have documented what syntax you were using to test importing these? (will save me the time of figuring out what opts i want)22:40
*** amotoki has quit IRC22:41
*** mriedem1 is now known as mriedem22:41
*** jkilpatr has joined #openstack-infra22:42
fungithough i guess only the opts in http://git.openstack.org/cgit/openstack-infra/storyboard/tree/storyboard/migrate/cli.py are probable to be relevant22:42
SotKfungi: yes, one moment22:43
fungithe only one i'm really wondering about is --auto-increment22:43
SotKI didn't use that in my tests22:44
*** krtaylor has quit IRC22:45
*** masber has joined #openstack-infra22:45
SotKI was doing `tox -e venv "storyboard-migrate --config-file ./etc/storyboard.conf --from-project monasca --to-project openstack/monasca"`22:46
SotKI'd imagine you don't want the tox bit22:47
*** Guest8383 is now known as rook-ugh22:47
fungiaha, i was wondering if it knew where to find the global config22:47
fungilooks like ours is /etc/storyboard/storyboard.conf (readable only by the www-data user)22:48
bkeropabelanger: Would you happen to know anything about http://paste.openstack.org/show/603514/ ?22:49
clarkbbkero: its the you have old setuptools and need to update setuptools first before anything else symptom22:50
bkeroclarkb: It's failing during venv/bin/pip install -U setuptools22:50
clarkbbkero: so pip install -U setuptools && pip install foo22:51
pabelangerwas going to say22:51
bkeroclarkb: http://paste.openstack.org/show/603516/22:51
openstackgerritThomas Herve proposed openstack-infra/project-config master: Make Zaqar Swift gate voting  https://review.openstack.org/44771922:51
bkeroClean centos7 host22:51
*** ym200 has quit IRC22:51
*** mozart89 has quit IRC22:51
clarkboh nice, how old is your virtualenv?22:52
bkeroa minute or two22:52
clarkbI mean the version of virtualenv22:52
pabelangerbkero: do you have six installed?22:52
clarkbvirtualenv bundles setuptools and maybe its old enough that even updating setuptools can't fix things22:53
*** mtanino has quit IRC22:53
*** dkehn has quit IRC22:53
bkerohttp://paste.openstack.org/show/603518/22:53
bkeroThere's a clean example: ^22:53
*** dkehn has joined #openstack-infra22:53
bkeropabelanger: yes22:53
bkeroFrom yum22:54
clarkbbkero: what does virtualenv --version say?22:54
bkero1.10.122:54
clarkbsix from yum shouldn't affect the code executing within the virtualenv22:54
*** EricGonczer_ has joined #openstack-infra22:54
bkeroNope, it shouldn't22:54
*** ym200 has joined #openstack-infra22:55
clarkbhttps://pypi.python.org/pypi/virtualenv/15.1.0 is current so what you ahve is incredibly old22:55
bkero*shrug* It's what's in centos722:55
clarkbwondering if you have to do a multi step upgrade to get from old venv setuptools to current setuptools22:55
* bkero doesn't understand how this isn't broken everywhere, just for him22:56
clarkbbecause the rest of us don't use 4 yaer old setuptools to install stuff22:57
clarkbyou have setuptools 0.9.822:57
bkeroWhere does the package come from for the centos7 tests though?22:57
clarkbpip22:57
fungiSotK: oh, hah! the monasca team doesn't actually have a repo named "monasca" so i'll set these to go to openstack/monasca-api which is the repo for their primary service (can always update it in the db later if these stories need to be moved to a different one)22:57
clarkber pypi22:57
clarkbvia get-pip -> pip -> package install22:57
*** kaisers has quit IRC22:57
bkeroI'm talking the python-virtualenv package22:57
clarkbyes that22:57
pabelangerbkero: look at install_puppet.sh in system-config22:57
SotKfungi: heh, yeah that seems sensible enough22:57
clarkbwe don't use distro virtuaelnv22:57
pabelangerthat is how we bootstrap centos nodes22:57
clarkbto avoid these problems22:58
*** marst has quit IRC22:58
clarkbyou may be able to do pip install six && pip install -U setuptools && pip install $rest22:59
bkeroThat got further23:00
bkerohttp://paste.openstack.org/show/603520/23:00
clarkbfor the second one I think you'll need an intermediate setuptools update23:00
clarkbpip insatll -U setuptools==someolderversion && pip install -U setuptools23:01
bkerosuch bootstrap23:01
clarkbbut I don't know what that older version is23:01
clarkbOr just use pypi provided virtauelvn23:01
pabelangerbkero: https://github.com/pypa/setuptools/blob/master/setup.py#L16523:01
pabelangeryou need those dependencies now23:01
fungiSotK: hooray, it's importing!23:01
clarkboh packaging is its own thing I thought it was bundled23:01
bkeropabelanger: Okay, installed the dependencies and it actually worked this time.23:02
pabelangerclarkb: they switch it out recently23:02
* bkero was trying to reproduce https://bugs.launchpad.net/tripleo/+bug/1674067 and ran into this issue23:02
openstackLaunchpad bug 1674067 in tripleo "newton: tripleo-quickstart jobs don't build packages from stable/newton" [Critical,Triaged] - Assigned to Attila Darazs (adarazs)23:02
fungiSotK: i've probably got it covered at this point and will continue to monitor and trigger additional repo imports throughout my evening. thanks for getting this working and tested!23:02
bkeroMaybe the simplest solution would be to run install-puppet.sh on the host23:02
SotKfungi: great! no problem, I'm happy we're finally beginning to migrate folks23:04
*** beslemon has quit IRC23:04
* SotK will likely be around for a little longer anyway so just shout if anything goes wrong weirdly23:04
fungiwill do--thanks again!23:05
pabelangerbkero: at a minimum, you should read the comments in install_puppet.sh for centos23:06
*** ym200 has quit IRC23:08
*** rbrndt has quit IRC23:08
bkeropabelanger: Yep, got it. Thank you for your knowledge and suggestions.23:10
*** marst has joined #openstack-infra23:10
openstackgerritThomas Goirand proposed openstack-infra/project-config master: Add /usr/share/puppet/modules  https://review.openstack.org/44772323:11
zigopabelanger: What do you think of this patch?23:11
pabelangerzigo: it shouldn't be needed, we install our modules into /etc/puppet/modules23:12
pabelangerdoes that fix things for you?23:12
*** kaisers has joined #openstack-infra23:12
zigopabelanger: Reading the logs, packages are installed in /usr/share/puppet/modules23:13
pabelangerzigo: okay, that is the wrong location then. Possible something has changed23:13
pabelangerzigo: mind sharing the build log23:13
zigopabelanger: See this: http://sid.gplhost.com/os/build-image.log.txt and search for: Cloning into '/usr/share/puppet/modules/snmpd'...23:13
pabelangerYa, that is wrong23:14
pabelangerhttp://nb01.openstack.org/dib.debian-jessie.log23:14
*** andreww has quit IRC23:14
pabelanger/etc/puppet/modules/snmpd23:14
pabelangerso, something install_modules.sh is not right23:15
zigoI'm not sure how I can check my changes, since "prepare-node" is in project-config that seemingly is git cloned in the chroot.23:15
*** pradk has quit IRC23:15
pabelangerzigo: puppet config print modulepath | cut -d ':' -f 123:15
pabelangerwhat is the value for sid23:15
zigopabelanger: /usr/share/puppet/modules23:16
pabelangerk23:16
pabelangerthat is the issue23:16
pabelangerit is /etc/puppet/modules for other nodes23:16
zigopabelanger: Ah, good we found it together then! :P23:17
zigopabelanger: Let me try to fix...23:17
zigopabelanger: Should I do something differently if running Stretch and beyond, or just hardcode /etc/puppet/modules ?23:18
openstackgerritThomas Goirand proposed openstack-infra/system-config master: Do not install from puppetlab for deb  https://review.openstack.org/44581323:20
* zigo reruns the script...23:21
*** vsaienko has quit IRC23:23
*** gildub has joined #openstack-infra23:24
*** ym200 has joined #openstack-infra23:28
ianychoiclarkb, with some discussions and my thinking on Zanata, I think it is the time to remove the old production instance for translate.o.o (it seems that https://bugs.launchpad.net/tripleo/+bug/1674011 is not related to the Zanata upgrade)23:32
openstackLaunchpad bug 1674011 in tripleo "String upload job failing with "missing end-quote"" [High,In progress] - Assigned to Julie Pichon (jpichon)23:32
*** masber has quit IRC23:32
clarkbianychoi: great to hear it. Its a bit late in my day but I will put it on my list for tomorrow23:32
*** rhallisey has joined #openstack-infra23:32
*** ym200 has quit IRC23:33
ianychoiclarkb, yep fine :) Also Can I switch openstackid-dev to openstack-id for a while in translate-dev.o.o? To do it, the agreement from smarcet might be better?23:33
ianychoi(We can also discuss it tomorrow. Have a good evening and dinner today, clarkb!)23:35
*** jamielennox is now known as jamielennox|away23:36
*** vhosakot has joined #openstack-infra23:36
clarkbianychoi: yes I think we should hear back from smarcet but I am fine wiht it23:36
ianychoiThanks a lot :)23:37
*** s-shiono has joined #openstack-infra23:37
*** s-shiono_ has joined #openstack-infra23:37
*** ym200 has joined #openstack-infra23:38
*** jamielennox|away is now known as jamielennox23:40
*** gyee_ has joined #openstack-infra23:41
*** s-shiono has quit IRC23:41
*** gyee has quit IRC23:43
*** ym200 has quit IRC23:43
*** dave-mccowan has joined #openstack-infra23:43
*** vhosakot has quit IRC23:45
*** vhosakot has joined #openstack-infra23:46
*** _Cyclone_ has joined #openstack-infra23:50
* SotK disappears23:50
*** gongysh has joined #openstack-infra23:55
zigopabelanger: It didn't fix it. :(23:55
*** ym200 has joined #openstack-infra23:56
zigoTo me it is as if no module were downloaded or something.23:57
*** tphummel has quit IRC23:57
*** ym200 has quit IRC23:57
kevinbentoni have two patches that seem to refuse to go to gate23:58
kevinbentonhttps://review.openstack.org/#/c/446204/23:58
kevinbentonhttps://review.openstack.org/#/c/446152/23:58
kevinbentonis there a way to tell what might be upsetting the gate?23:58
*** ym200 has joined #openstack-infra23:58
zigoYeah, right, with my patch, no puppet modules were cloned... :/23:59

Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!