Friday, 2016-11-11

*** ayoung has quit IRC00:04
*** chlong has joined #tripleo00:12
*** dmacpher has quit IRC00:21
*** cdearborn has quit IRC00:25
openstackgerritBob Fournier proposed openstack/os-net-config: Stop dhclient in os-net-config if interface is not configured for DHCP  https://review.openstack.org/39644600:25
dsneddonbfournie++00:26
*** colonwq has joined #tripleo00:31
*** jerrygb has joined #tripleo00:34
*** dmacpher has joined #tripleo00:34
*** jerrygb has quit IRC00:38
*** limao has joined #tripleo00:39
*** ayoung has joined #tripleo00:45
openstackgerritGabriele Cerami proposed openstack-infra/tripleo-ci: POC: Initial transition to quickstart setup  https://review.openstack.org/39645100:45
openstackgerritBob Fournier proposed openstack/os-net-config: Stop dhclient in os-net-config if interface not set for DHCP  https://review.openstack.org/39644600:54
openstackgerritDan Sneddon proposed openstack/os-net-config: Stop dhclient in os-net-config if interface is not configured for DHCP  https://review.openstack.org/39644601:28
openstackgerritSagi Shnaidman proposed openstack-infra/tripleo-ci: TEST: DONT RECHECK: periodic jobs  https://review.openstack.org/35921501:29
*** jkilpatr has quit IRC01:35
*** tiswanso has joined #tripleo01:38
*** tiswanso has quit IRC01:38
*** tiswanso has joined #tripleo01:39
openstackgerritBob Fournier proposed openstack/os-net-config: Stop dhclient in os-net-config if interface not set for DHCP  https://review.openstack.org/39644601:43
openstackgerritSteve Baker proposed openstack/instack-undercloud: Configure undercloud docker for insecure registry  https://review.openstack.org/39646001:46
*** bfournie has quit IRC01:49
*** fzdarsky|afk has quit IRC02:08
*** limao has quit IRC02:08
*** limao has joined #tripleo02:09
*** chlong has quit IRC02:11
*** chandankumar has joined #tripleo02:17
*** chandankumar has quit IRC02:18
*** tiswanso has quit IRC02:22
*** tiswanso has joined #tripleo02:22
*** anton has quit IRC02:30
*** jerrygb has joined #tripleo02:34
*** jerrygb_ has joined #tripleo02:37
*** jerrygb has quit IRC02:39
*** bfournie has joined #tripleo02:49
openstackgerritBob Fournier proposed openstack/os-net-config: Stop dhclient in os-net-config if interface not set for DHCP  https://review.openstack.org/39644602:53
*** jerrygb has joined #tripleo02:54
*** jerrygb_ has quit IRC02:56
*** jerrygb has quit IRC03:00
*** tzumainn has quit IRC03:17
*** mhenkel has quit IRC03:17
*** ebalduf has joined #tripleo03:21
openstackgerritRedHat RDO CI proposed openstack/tripleo-heat-templates: GATE TEST, please ignore  https://review.openstack.org/36544903:30
*** mhenkel has joined #tripleo03:48
*** sudipto has joined #tripleo04:10
*** sudipto_ has joined #tripleo04:10
*** fzdarsky|afk has joined #tripleo04:10
*** tiswanso has quit IRC04:11
*** mhenkel has quit IRC04:12
*** jerrygb has joined #tripleo04:20
*** mhenkel has joined #tripleo04:22
*** jerrygb has quit IRC04:23
*** jerrygb has joined #tripleo04:23
*** bana_k has joined #tripleo04:24
*** jerrygb has quit IRC04:28
*** mhenkel has quit IRC04:33
*** bana_k has quit IRC04:35
*** numans has joined #tripleo04:45
*** fragatin_ has joined #tripleo04:53
*** fragatina has quit IRC04:57
*** fragatin_ has quit IRC04:57
*** fragatina has joined #tripleo05:00
*** jerrygb has joined #tripleo05:01
*** fragatina has quit IRC05:01
*** jerrygb has quit IRC05:02
*** Vijayendra has joined #tripleo05:07
*** mhenkel has joined #tripleo05:10
*** rlandy has quit IRC05:11
*** yamahata has quit IRC05:21
*** thrash is now known as thrash|g0ne05:22
*** masco has joined #tripleo05:24
*** fragatina has joined #tripleo05:37
*** nyechiel has joined #tripleo05:38
*** coolsvap has joined #tripleo05:45
*** rajinir has quit IRC05:46
*** florianf has joined #tripleo05:52
*** mhenkel has quit IRC05:54
*** nyechiel has quit IRC05:59
*** chandankumar has joined #tripleo06:21
*** redhatkj has quit IRC06:22
openstackgerritAparna proposed openstack/diskimage-builder: element: proliant-tools: Update hpssacli to ssacli  https://review.openstack.org/39650406:30
*** jaosorior has joined #tripleo06:32
*** redhatkj has joined #tripleo06:33
*** hewbrocca_afk is now known as hewbrocca06:37
*** lmiccini has joined #tripleo06:38
*** hewbrocca is now known as hewbrocca_afk06:38
*** hewbrocca_afk is now known as hewbrocca06:39
*** abehl has joined #tripleo06:41
*** ealcaniz has joined #tripleo06:43
*** aufi has joined #tripleo06:44
*** pmannidi has quit IRC06:47
*** oshvartz has joined #tripleo06:48
*** iranzo has joined #tripleo06:48
*** iranzo has quit IRC06:48
*** iranzo has joined #tripleo06:48
jaosoriortherve: around yet?06:59
*** yamahata has joined #tripleo06:59
*** udesale has joined #tripleo06:59
*** tesseract has joined #tripleo07:03
*** tesseract is now known as Guest760507:03
*** b00tcat` has joined #tripleo07:05
*** rasca has joined #tripleo07:06
*** Alex___1234 has quit IRC07:09
*** rcernin has joined #tripleo07:11
*** hewbrocca is now known as hewbrocca_afk07:12
*** rasca has quit IRC07:18
*** rasca has joined #tripleo07:18
*** ebalduf has quit IRC07:21
openstackgerritJuan Antonio Osorio Robles proposed openstack/instack-undercloud: Configure keystone ec2 conf for heat behind haproxy  https://review.openstack.org/39651607:21
openstackgerritJuan Antonio Osorio Robles proposed openstack/instack-undercloud: Move glance_api_servers conf for nova to hiera  https://review.openstack.org/39651707:21
openstackgerritJuan Antonio Osorio Robles proposed openstack/instack-undercloud: Pass keystone configuration via hiera  https://review.openstack.org/39651807:21
*** jprovazn has joined #tripleo07:21
*** oshvartz has quit IRC07:25
*** ealcaniz has quit IRC07:27
*** jaosorior has quit IRC07:31
openstackgerritJulie Pichon proposed openstack/instack-undercloud: Increase Mistral Task Size limit  https://review.openstack.org/39652307:34
*** ealcaniz has joined #tripleo07:34
openstackgerritJulie Pichon proposed openstack/python-tripleoclient: Reset the parameter_defaults between deployments via the CLI  https://review.openstack.org/39652407:36
*** anton has joined #tripleo07:40
*** oshvartz has joined #tripleo07:41
*** yolanda has quit IRC07:41
*** yolanda has joined #tripleo07:41
*** pmannidi has joined #tripleo07:50
*** ebarrera has joined #tripleo07:54
*** gbarros has quit IRC07:56
*** jlinkes has joined #tripleo08:00
*** tremble has joined #tripleo08:00
*** jaosorior has joined #tripleo08:02
*** zoli|gone is now known as zoli|wfh08:03
*** phpcodemonkey has joined #tripleo08:04
jaosoriorcan anybody review this commit for CI? https://review.openstack.org/#/c/396056/08:05
jaosoriormarios: hey dude, could you check this one out? https://review.openstack.org/#/c/392473/ it's a quick one :D08:06
*** ccamacho has joined #tripleo08:08
jaosoriorccamacho: hey, if you have time today could you check these TLS patches? https://review.openstack.org/#/c/392090/ https://review.openstack.org/#/c/392095/08:10
ccamachohey jaosorior morning, sure let me add them to my list :)08:11
mariosjaosorior: done08:12
jaosoriormarios: thanks dude08:13
*** shardy has joined #tripleo08:17
*** hjensas has quit IRC08:18
jaosoriorshardy: any idea what would cause this http://paste.openstack.org/show/588907/ to appear all over the cfn api?08:18
jaosoriortherve: ^^08:21
*** athomas has joined #tripleo08:22
shardyjaosorior: could be a number of things, but basically the heat ec2token middleware takes the request and signature, and asks the keystone ec2tokens extension to validate it08:24
shardythat is failing, either because we've mangled something in the URL so the signature is now wrong, or because the ec2tokens API is somehow broken08:25
shardyjaosorior: does the switch to the httpd API involve any rewriting of the URL we get in the API vs what the client requests?08:26
shardythe signature calculation requires the path etc to be *exactly* the same, or the signatures won't match08:26
jaosoriorshardy: it doesn't08:26
jaosoriorshardy: is the signature passed via a header or something of the sort?08:29
shardyjaosorior: I think we're still using the v2 CFN API format, so it's just part of the query string08:30
jaosoriorshardy: how does it change in further versions?08:30
*** ccamacho has quit IRC08:31
*** jpich has joined #tripleo08:31
shardyjaosorior: I'd probably put some debug in here:08:31
shardyhttps://github.com/openstack/python-keystoneclient/blob/master/keystoneclient/contrib/ec2/utils.py#L8008:31
shardythen compare the credentials dict during the create with that used during the validation08:32
openstackgerritJulie Pichon proposed openstack/puppet-tripleo: Persist the UI config file  https://review.openstack.org/38859208:32
*** amoralej|off is now known as amoralej08:34
shardy(that library is used by both heat to calculate the signature, and keystone to validate it, so all else being equal they should match)08:34
*** hjensas has joined #tripleo08:37
openstackgerritJuan Antonio Osorio Robles proposed openstack/instack-undercloud: Move glance_api_servers conf for nova to hiera  https://review.openstack.org/39651708:38
openstackgerritJuan Antonio Osorio Robles proposed openstack/instack-undercloud: Pass keystone configuration via hiera  https://review.openstack.org/39651808:38
*** jpena|off is now known as jpena08:38
jaosoriorshardy: just to be clear, this generate function is called from heat in the undercloud? or where?08:43
shardyjaosorior: heat and keystone on the undercloud both call it08:43
b00tcat`what is the difference between `config_settings` and `service_config_settings` used in the service YAML files? I don't quite get it :-/08:43
shardyheat calls it to calculate the signature08:43
*** phpcodemonkey has quit IRC08:43
shardyjaosorior: then later, when we signal back to heat, the ec2tokens middleware asks keystone's ec2tokens api to validate the signature08:44
shardywhich happens to use the exact same keystoneclient code08:44
shardyin both cases we provide details about the request, and get a hash back08:44
shardyso something about the request we give to keystone when the signal happens must be wrong08:44
*** ohamada has joined #tripleo08:45
shardyjaosorior: you could also do a minimal reproducer with a heat stack that only contains one OS::Heat::SoftwareDeployment resource and nothing else08:45
shardythat will create a signature, and you can hit the URL it exposes via curl08:46
shardywhich is essentially all TripleO is doing08:46
shardyb00tcat`: config_settings is data we deploy on the node where the service is running, e.g in services/heat-api.yaml it's deployed on the nodes running heat-api08:47
*** abregman has joined #tripleo08:47
shardyb00tcat`: service_config_settings is used when e.g heat-api needs the node running keystone to do something, e.g create a user or similar08:47
shardyb00tcat`: so in that case we wire service_config_settings: {keystone: foo ...08:48
shardyto the node running keystone08:48
shardywhich isn't necessarily the same one that's running e.g heat08:48
shardyb00tcat`: we do need better docs on this, thanks for the reminder08:48
openstackgerritDougal Matthews proposed openstack/python-tripleoclient: Format the nodes list in openestack overcloud delete node  https://review.openstack.org/39508308:52
openstackgerritJiri Tomasek proposed openstack/tripleo-specs: GUI Deployment Configuration improvements  https://review.openstack.org/39336508:53
*** paramite has joined #tripleo08:56
jaosoriorshardy: ok, trying out the minimal reproducer08:56
Ngmorning folks08:57
*** abregman has quit IRC08:58
Ngcan we try and get bnemec's puppet-lint fix landed? It's blocking zuul from merging a few backports atm. It's https://review.openstack.org/#/c/396419/108:58
Ng(well, it's blocking anything from landing in puppet-tripleo, but I know of at least two backport patches that would be very welcome if they could land before an internal deadline today :)09:00
Ngjaosorior: :)09:00
shardyNg: done!09:00
Ngawesome, thank you both!09:00
openstackgerritMerged openstack/tripleo-heat-templates: Add missing Barbican endpoint from tls-everywhere environment  https://review.openstack.org/39247309:00
* shardy wonders why we didn't catch that in CI before it landed09:01
*** zoli|wfh is now known as zoli|doctor09:01
Ngshardy: I think it may be that a package was upgraded, and the linter got slightly stricter09:01
shardyah, I guess that would explain it09:01
*** b00tcat` has quit IRC09:05
jaosoriorshardy: soo... how do I get that URL from the deployment?09:05
*** mcornea has joined #tripleo09:07
*** b00tcat` has joined #tripleo09:07
shardyjaosorior: ah, I forgot, you'll need either a server or a dummy resource acting as a server:09:07
shardyhttps://etherpad.openstack.org/p/noop-softwareconfig09:08
shardyyou can cut/paste from that I think09:08
b00tcat`thanks for the answer shardy09:08
shardysee the deploy_signal_id in the resource metadata for the dummy server09:08
jaosoriorI deployed it on a server09:08
shardyjaosorior: Ok then you can just do heat resource-metadata thestack theserver_resource_name09:08
*** bnemec has quit IRC09:09
jaosoriorok, deploy_signal_id looks like this http://172.29.172.1:8000/v1/signal/arn%3Aopenstack%3Aheat%3A%3A039a7d5c7f214d55b4fe90855dca7327%3Astacks%2Fwalrus%2Fc919935a-e241-4a6f-a58b-6323bd67d0fa%2Fresources%2Fdeploymentthing?Timestamp=2016-11-11T08%3A56%3A36Z&SignatureMethod=HmacSHA256&AWSAccessKeyId=f94159185bb14e189c8f882239a2c3a4&SignatureVersion=2&Signature=ulsDqSkgM%2FCudiIETkxNEB4374%2FdoJr2CMwaSNF4ROo%3D09:09
jaosoriorwow09:10
openstackgerrityolanda.robla proposed openstack/tripleo-quickstart: Create directories with root  https://review.openstack.org/38489209:10
jaosoriorshardy: http://paste.openstack.org/show/588912/09:10
jaosoriorright so it's an issue with the URL apparently09:11
shardyjaosorior: did you put quotes around the entire URL?09:11
jaosoriorI didn't09:11
shardycurl -X POST "http://172.29.172.1:8000/v1/s ..."09:11
*** bnemec has joined #tripleo09:12
shardyhttps://github.com/openstack/tripleo-image-elements/blob/master/elements/os-refresh-config/os-refresh-config/post-configure.d/99-refresh-completed#L3209:12
jaosoriornow that I did I got this: http://paste.openstack.org/show/588913/09:12
shardyOk, so looks like you reproduced your issue09:12
shardyso you can add details to the ec2signer stuff and compare the data used to calculate the signature on stack create09:13
shardywith that used when doing the curl call09:13
jaosoriornot entirely sure what that means yet, if it's an actual authorization issue or that the signature didn't match09:13
shardythe heat logs and some keystone debug will tell you I think09:13
shardyI suspect the move to httpd has somehow subtly changed the data we pass to ec2Signer09:14
jaosoriorcrap09:18
jaosorioradding debug logs to keystone middleware does nothing09:18
jaosorior* keystoneclient09:18
*** mcornea has quit IRC09:19
*** mcornea has joined #tripleo09:20
*** aufi_ has joined #tripleo09:21
*** aufi has quit IRC09:21
*** lucas-afk is now known as lucasagomes09:22
shardyjaosorior: did you restart heat-engine and keystone?09:23
jaosoriorI did09:24
shardyyou can even kill heat-engine and run it on the CLI and grep for your debug09:24
jaosoriornow I'm trying to create a logging object and write that log file to my home dir09:24
jaosoriorlets see if that works09:24
*** derekh has joined #tripleo09:25
shardyhttps://github.com/openstack/heat/blob/master/heat/engine/resources/signal_responder.py#L17309:25
shardythat's where heat calls keystoneclient FYI09:25
*** derekh has quit IRC09:29
*** gfidente has joined #tripleo09:31
jaosoriorshardy: ok, I think I got it09:37
jaosoriorSo this is the actual signature used q61gKkwVqhHxRd3dfl2FfaBPf3/8t2doLc1LeO/7I0s=09:37
gfidentemwhahaha, you around?09:38
gfidenteany clue why this is failing the tests only for puppet 4.5?09:38
gfidentehttps://review.openstack.org/#/c/395785/09:38
Nggfidente: https://review.openstack.org/#/c/396419/1 :)09:40
Ng+A, just waiting for zuul to grant it access to the underworld, and then we can land puppet stuff again09:40
jaosoriorshardy: looks like this http://paste.openstack.org/show/588918/09:40
gfidenteNg, that fixes the lint issue indeed09:40
gfidentebut I was looking at the unit tests instead09:41
jaosoriorgfidente: it'll be fixed in around ~20 min09:41
jaosorioroh, if that's another issue I don't know09:41
Nggfidente: oh, doh, sorry09:41
*** rbrady is now known as rbrady-afk09:41
gfidenteyeah it looks like the unit tests are passing for puppet < 409:42
gfidenteat least for that submission09:42
shardyjaosorior: ack, so yeah now you need to compare the inputs to the signature calculation, one of them must be different during the API request09:43
shardythe "/" in the signature looks bogus to me?09:43
openstackgerrityolanda.robla proposed openstack/tripleo-quickstart: Create directories with root  https://review.openstack.org/38489209:43
jaosoriorshardy: it's what appeared in the logs :/09:44
*** ccamacho has joined #tripleo09:45
*** derekh has joined #tripleo09:47
*** openstackgerrit has quit IRC09:48
*** openstackgerrit has joined #tripleo09:48
shardyAnyone feel like approving https://review.openstack.org/#/c/396317/ ?09:49
shardyAlso https://review.openstack.org/#/q/Id240709e256b057bbdff038aefc7fcacb0d339b8,n,z09:49
*** abregman has joined #tripleo09:50
shardymarios, matbu: Hey, spotted a possible issue re https://review.openstack.org/#/c/395234/409:52
openstackgerritMerged openstack/puppet-tripleo: Fix puppet lint failure  https://review.openstack.org/39641909:53
shardyI'm not sure we can unconditionally enable external ceph there?09:53
b00tcat`shardy: this blogpost of yours where you explain composable services points to the default ControllerServices list at https://github.com/openstack/tripleo-heat-templates/blob/master/overcloud.yaml#L356 however this link is a dead end and grepping by "ControllerServices" didn't give any result back09:53
b00tcat`where can I find the default services list?09:53
jaosoriorany +As for this patch adding scenario004 to our tests? https://review.openstack.org/#/c/396056/09:53
shardyhttps://github.com/openstack/tripleo-heat-templates/blob/master/roles_data.yaml#L2309:54
shardyb00tcat`: ah, it moved after I wrote the post to roles_data.yaml09:54
shardywe now template in the defaults with jinja2, which was needed for the custom-roles implementation (explained in another more recent blog post)09:54
d0ugalWhat does "workload cloud stack" mean? https://github.com/openstack/tripleo-common/blob/master/workbooks/deployment.yaml#L10109:54
b00tcat`shardy: ok I will check it out09:55
shardyd0ugal: AFAICS it means "overcloud"09:55
d0ugalshardy: aha, thanks. I might change it to that :)09:56
shardyyeah, or just remove it09:56
openstackgerrityolanda.robla proposed openstack/tripleo-quickstart: Create directories with root  https://review.openstack.org/38489209:57
pandamaster undercloud is borked http://logs.openstack.org/periodic/periodic-tripleo-ci-centos-7-ovb-nonha/9521bfb/logs/undercloud/var/log/undercloud_install.txt.gz#_2016-11-11_06_22_11_00010:02
pandaopening bug10:02
openstackgerritDougal Matthews proposed openstack/tripleo-common: Only start the deploy if the Heat stack isn't already in progress  https://review.openstack.org/39656710:02
yolandahi, good morning. Can I get reviews on https://review.openstack.org/394426 ?10:02
ccamachoshardy jaosorior.. Just to add some more info about the lint snag, we have upgraded to puppet lint 2.0.2. Here are the tests (https://review.openstack.org/#/q/branch:master+topic:puppet-lint_check) but not sure how I missed puppet-tripleo10:03
ccamachoLanded yesterday night..10:03
ccamachoBut missed that fix.. :( sorry for that10:03
mariosshardy: thanks unset +2 as well and added comment, should we go back to v1 approach then matbu https://review.openstack.org/#/c/395234/410:04
shardymarios: I don't think the addition to the hierarchy works, so we need to document including environments/puppet-ceph-external.yaml on upgrade perhaps?10:05
*** nijaba has quit IRC10:06
matbushardy: marios hey guys, the addition of the hiera file works, but it's a wrong fix10:06
mariosshardy: but the error (about Could not find data item ceph::profile::params::client_keys in any Hiera data ) happens on compute nodes https://bugs.launchpad.net/tripleo/+bug/1640328 and as you said perhaps they don't have ceph or even external ceph at that10:06
openstackLaunchpad bug 1640328 in tripleo "[Upgrade] Could not find data item ceph::profile::params::client_keys in any Hiera data" [High,In progress] - Assigned to mbu (mat-bultel)10:06
*** abregman is now known as abregman|nb10:07
shardymarios: Yeah on compute it should be provided by OS::TripleO::Services::CephClient10:07
*** nijaba has joined #tripleo10:07
*** nijaba has quit IRC10:07
*** nijaba has joined #tripleo10:07
shardyin the non-external case10:07
*** pmannidi has quit IRC10:08
shardymatbu: yeah I think it works because it's reading old hieradata, but that would mean any future config changes are ignored10:09
matbuactually i got another issue result of fixing this bug. I'm off today but i'll try to update this review during the week end10:09
mariosmatbu: ok thanks matbu forgot you are off today enjoy10:09
shardymatbu: ack Ok, we can revisit on Monday, thanks!10:09
matbushardy: yes, btw, i was wondering if it's possible to remove the old hiera files ?10:09
shardymatbu: Yeah we should, but currently don't10:10
matbushardy: i guess so, but10:10
shardymatbu: as part of the move to using dprince's hiera hook, I'd like to be able to purge all hieradata, and write it freshly each deploy10:10
matbushardy: k, it would be better because, in the upgrade case it's a bit dangerous i think10:10
openstackgerritGiulio Fidente proposed openstack/puppet-tripleo: Normalize civetweb binding address if IPv6  https://review.openstack.org/39578510:10
shardymatbu: yeah - it probably will work because of the hierarchy ordering, but I agree it's at best confusing, and potentially a source of bugs10:11
matbushardy: yep10:11
shardyI'll raise a bug so we can track it10:12
matbushardy: cool thanks10:12
jaosoriorshardy: well... still don't figure out the issue... the signatures look fine and stuff doesn't seem to be changing http://paste.openstack.org/show/588927/ any idea where the AWSAccessKeyId comes from?10:15
openstackgerritMerged openstack/tripleo-heat-templates: Correct opencontrail hiera interpolation key  https://review.openstack.org/39329210:17
shardyjaosorior: yes it's also provided from the keystone ec2tokens API10:17
shardyheat creates an ec2 keypair, then uses the key/secret for signing10:17
openstackgerritMerged openstack/tripleo-heat-templates: Correct opencontrail hiera interpolation key  https://review.openstack.org/39334010:17
shardysame data is used by keystone as credentials for validating the signature10:17
shardyif the signatures match, I don't see why keystone would nack the validation of the signature?10:18
shardyall it does is compare the two values10:18
openstackgerritMartin Mágr proposed openstack/tripleo-heat-templates: Use default Sensu redact  https://review.openstack.org/39484710:18
jaosoriorshardy: cause at some point I see some other AWSAccessKeyId being used for that same resource10:19
shardyhttps://github.com/openstack/heat/blob/master/heat/api/aws/ec2token.py#L22210:19
shardyyou might want to comment out the except there and let the error blow up so you can inspect the backtrace10:19
openstackgerritTomas Sedovic proposed openstack/tripleo-docs: Add tripleo-validations user documentation  https://review.openstack.org/39619710:20
shardyjaosorior: the AWSAccessKeyId is associated with each deployment, not the server10:20
shardyso in a test stack with one deployment, the access key should always be the same10:21
jaosoriorstrange10:21
jaosoriorthere is onl;y one deployment10:21
shardyIf the access key on validation is different that definitely explains it10:21
shardybut I don't understand how that happens, because heat calculates the signature, and pushes metadata right into the server10:21
shardythere's nowhere for the access key to get mangled there AFAICT10:21
jaosoriorwhen the scripts to depoy over apache were written, was there any more documentation about it?10:23
shardyjaosorior: not sure tbh, you may want to git blame the wsgi stuff from the heat tree10:25
shardyseems like you're close to figuring it out tho - you just need to know exactly why it's hitting that execpt block10:25
shardycould even be we've got an unrelated Key/AttributeError tripping it10:25
mariosshardy: newton backport please https://review.openstack.org/#/c/395460/ green ci run and master merged last night10:26
mariosshardy: (if/when you get a chance)10:26
shardymarios: will do, wow quite a big patch10:29
mariosshardy: yeah it is a scary looking patchh. there is discussion on master. it is just because it shifts everything over by one step to isolate the ng ha migration into step 110:29
shardymarios: just to clarify - the scripts are run in sequence right, so I can copy the logic out from those steps and into my composable PoC which runs per-step over all roles?10:30
mariosshardy: so scary looking but actually not adding anything just shifting10:30
mariosshardy: yeah we do it with depends on as usual like https://review.openstack.org/#/c/395460/3/extraconfig/tasks/major_upgrade_pacemaker.yaml10:31
mariosshardy: i mean the steps10:31
shardymarios: Ack, thanks, so it will translate directly into the steps I defined10:31
shardymarios: jistr was asking for names instead of step numbers, what are your thoughts on that?10:31
shardyI'm leaning towards leaving the steps, as it's more consistent with the deployment steps10:31
shardyand closer to this existing implementation10:31
mariosshardy: so bandini made a good point on one of the reviews (maybe this one... sec)10:31
shardywe could make it configurable, e.g have a list of steps for each role in roles_data.yaml10:32
shardyso adding a new step wouldn't be a big deal10:32
shardyperhaps do 10, 20, 30 to allow for that10:32
shardyOr, we could use names and define the ordering in a similar way10:32
openstackgerritRaoul Scarazzini proposed openstack/tripleo-quickstart: Add swift and block to default node flavors  https://review.openstack.org/39657710:33
mariosshardy: so names alone don't work.. we need the numbers too so we can enforce sequencing10:34
shardymarios: yeah, but we could use names combined with a list that determines ordering10:34
mariosshardy: but then if we have name_number we would have to keep renaming things (this was the gist of bandini comments https://review.openstack.org/#/c/388649/ )10:34
shardyamounts to the same thing, I just wanted to clarify the consensus re which folks prefer10:34
shardybefore doing a second pass on my PoC patch10:35
mariosshardy: right if we do names plus sequence we may ave to keep renaming things. leaving gaps like 10/20/30 is probably going to take care of that concern though10:35
mariosshardy: so we don't have to rename 22KeystoneThing we just add 23AFterKeystoneThing10:35
mariosshardy: (sorry bandini comments at Oct 25 1:03 PM )10:36
*** tosky has joined #tripleo10:36
shardymarios: ack Ok we can go with 10/20/30 for now then10:37
*** limao has quit IRC10:37
shardyI'm thinking this will be statically defined per role in roles_data.yaml tho, do you think we need some way to allow services to plug in at arbitrary steps?10:38
*** yamahata has quit IRC10:38
shardymaking it dynamically decide the steps at runtime will be more work I think, although maybe possible10:38
jistrwhat's the advantage of 10/20/30 over names though?10:38
shardyjistr: Just that it makes the ordering a bit more obvious/explicit10:39
mariosshardy: 'plug in' to an overall upgrades workflow you mean? not clear what 'this' is either at tis point :)10:39
jistrok10:39
mariosjistr: the idea was to have names_number like INIT_10 or w/e10:39
mariosjistr: so both10:39
mariosjistr: the name bit is only for the humans though10:39
openstackgerritTomas Sedovic proposed openstack/tripleo-docs: Add tripleo-validations user documentation  https://review.openstack.org/39619710:40
shardymarios: by plug in I mean, if someone wants to add INIT_11 - is it reasonable to make them edit roles_data.yaml to add the new step?10:40
shardyor do we have to make it dynamic, e.g you just add INIT_11: foo to services/foo.yaml10:41
shardythen the step is automagically wired in10:41
shardythe former is what I've prototyped, the latter may be possible but will mean more work, and probably more expensive heat resources10:41
shardylike a ResourceChain of deployments or something10:41
shardythe other option is to roll up all the steps and drive the entire workflow via either mistral or ansible10:42
mariosshardy: so it depends what the steps are that we are defining in roles data. is it the overall upgrades workflow ? I thought we would be doing the latter, i.e. tagging upgrade config/steps in the servies directly that would then get combined10:42
shardyheat basically becomes an ansible playbook generator in that case10:42
mariosshardy: defining an overall start bit , including the stuff in order from all the services, and then any end bits.10:43
mariosshardy: yeah so i thought that's what we were describing in the spec so far anyway10:43
jistrhmm yea one potential advantage of having the numbers is that we don't have to list the steps upfront, and have them automatically generated. (Maybe we could go 11_init instead of init_11 to make the ordering super easy...)10:43
jistrhopefully if we go 00 - 99 we won't run out of numbers :D10:43
shardy000 - 999 ;)10:44
jaosoriorshardy: where is that stacktrace supposed to appear? I see no sign of it in any of the logs :/10:44
mariosshardy: i mean that the core idea was to decentralise the config so it lives in each service and then 'the tripleo tooling' would combine those into the script or scripts that  the operator would use to drive the upgrade10:44
shardyjaosorior: in the heat-api-cfn logs10:44
mariosshardy: i mean https://review.openstack.org/#/c/392116/ (i'll update that to point to your protoypes in the spec too)10:44
jaosoriornope10:45
b00tcat`why are parameters passed twice to THT services (from puppet-tripleo and THT)? i.e. https://github.com/openstack/puppet-tripleo/blob/master/manifests/profile/base/snmp.pp#L36 and https://github.com/openstack/tripleo-heat-templates/blob/master/puppet/services/snmp.yaml#L3810:45
mariosshardy: now having said all of that, i think at this point is 'whatever works' so if it is easier to define steps in roles_data for now do that10:45
shardyjaosorior: when we looked at a CI run for this recently, the move to httpd had somehow disabled all the cfn api logging10:45
shardydid that get fixed?10:45
jaosoriorshardy: it hadn't been disabled, it just didn't get any calls10:46
jaosoriorshardy: httpd does lazy initialization, so it hadn't been loaded yet10:46
jaosoriorthat was all10:46
jaosoriorso yeah, it's now getting logs10:46
openstackgerritSteven Hardy proposed openstack/tripleo-heat-templates: WIP prototyping composable upgrades with Heat+Ansible  https://review.openstack.org/39344810:47
shardymarios: ack, yeah my question is here (line 22)10:47
shardyhttps://review.openstack.org/#/c/393448/4/upgrade/major_upgrade_steps.j2.yaml10:47
shardyit's easier to use a fixed range of steps10:47
shardybut if we have to, we can allow it to be dynamic10:48
shardyI'll probably start with it fixed anyway10:48
arxcruzIs ooo support barbican ?10:48
mariosshardy: makes sense thanks shary will try review that some more today and update the spec later10:48
mariosshardy: we said we'd land the specs for O1 so should be doable by next week (spec i mean)10:49
shardymarios: ack, thanks - I'll re-review the spec later, and hopefully get another pass of the prototype up in the next few days10:49
shardyyeah hopefully we can agree the approach enough to land the spec on Monday10:49
*** athomas has quit IRC10:50
pandaany idea why this is not alerting ? https://bugs.launchpad.net/tripleo/+bug/164107610:50
openstackLaunchpad bug 1641076 in tripleo "CI: master job fail on undercloud install because of missing ceilometer-dbsync command" [Critical,Confirmed]10:50
jaosoriorshardy: nope, it doesn't crash10:56
*** athomas has joined #tripleo10:56
jaosoriorshardy: this is even the token it gets http://paste.openstack.org/raw/588932/10:57
shardyjaosorior: so why is it logging from https://github.com/openstack/heat/blob/master/heat/api/aws/ec2token.py#L223 ?10:58
openstackgerritDougal Matthews proposed openstack/tripleo-common: Only start the deploy if the Heat stack isn't already in progress  https://review.openstack.org/39656710:59
Ngwhat the heck, the puppet-lint fix landed, but my reverify failed with the old error?!11:00
*** owalsh has joined #tripleo11:00
jaosoriormakes no sense to me11:00
jaosoriorthen again, my knowledge of the heat code-base is no good either11:00
shardyjaosorior: well you seem to be really close now - it's going down the except (AttributeError, KeyError): path, why?11:01
shardyhttps://github.com/openstack/heat/blob/master/heat/api/aws/ec2token.py#L23711:01
shardyif it goes down that path, the API request will work11:02
*** dtantsur|afk is now known as dtantsur11:05
openstackgerritArx Cruz proposed openstack/tripleo-quickstart: Enable cinder-backup service on minimal pacemaker  https://review.openstack.org/39658711:06
*** d0ugal has quit IRC11:07
*** akrivoka has joined #tripleo11:09
*** ooolpbot has joined #tripleo11:10
ooolpbotURGENT TRIPLEO TASKS NEED ATTENTION11:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/164107611:10
*** ooolpbot has quit IRC11:10
openstackLaunchpad bug 1641076 in tripleo "CI: master job fail on undercloud install because of missing ceilometer-dbsync command" [Critical,Confirmed]11:10
jaosoriorshardy: by the way, noticed the ec2 endpoint was being configured for heat using the controller IP instead of the actual endpoint https://review.openstack.org/#/c/396516/1  not a big deal, just thought it would be good to change it11:11
jaosoriorshardy: aaand... it's weird, seems that for a tripleo deployment I'm seeing those authentication errors, but for the local deployment, that actually succeeds11:14
jaosoriors/local/minimal11:14
jaosoriorhowever, the deployment is still stuck11:16
openstackgerritArx Cruz proposed openstack/tripleo-quickstart: Configuring vcpus for undercloud, controller and compute nodes  https://review.openstack.org/39659311:17
openstackgerritDan Sneddon proposed openstack/os-net-config: Stop dhclient in os-net-config if interface not set for DHCP  https://review.openstack.org/39644611:18
shardyjaosorior: you can do the same debug by copying the URL from the server metadata for a failed TripleO deployment11:22
shardye.g curl the deploy URL11:22
jaosoriorshardy: well, the minimal deployment is still stuck :/ so either way it's an issue11:23
jaosoriorall I see from os-collect-config there is Nov 11 11:22:18 identity-base os-collect-config[2064]: /var/lib/os-collect-config/local-data not found. Skipping11:23
jaosoriorNov 11 11:22:18 identity-base os-collect-config[2064]: No local metadata found (['/var/lib/os-collect-config/local-data'])11:23
shardyjaosorior: did you set user_data_format: SOFTWARE_CONFIG on the server?11:24
*** hewbrocca_afk is now known as hewbrocca11:25
jaosorioryes11:25
shardyI'd focus on the curl call that didn't work11:25
shardyas by default os-collect-config will poll via the same API11:25
shardyso you would expect the server polling to be broken too11:25
jaosorioryeah, that's still not working11:26
shardyhttp://paste.openstack.org/show/588907/11:26
shardyit's going down that "AWS authentication failure." path, so we just need to know why (printing the exception would probably help give some clues)11:27
*** jprovazn has quit IRC11:28
jaosoriorshardy: yeah, the issue is that now that I went to using a more minimal install, I see authentication success in the logs11:28
*** abregman|nb is now known as abregman|afk11:28
jaosorioryes, using that URL, it still gets denied11:28
*** coolsvap has quit IRC11:30
* shardy is confused11:30
* jaosorior is confused and frustrated by this11:31
b00tcat`re. custom services: is there a thing like _get_nodes_by_role_ that I can use in tht? I mean, I would need some nodes with a certain role to be aware of themselves11:31
openstackgerritDan Sneddon proposed openstack/os-net-config: Stop dhclient in os-net-config if interface not set for DHCP  https://review.openstack.org/39644611:31
shardyjaosorior: In http://paste.openstack.org/show/588913/ you showed the curl request failing11:33
shardyso I'm confused when you now say the minimal example works11:33
*** Baggypants12000 has quit IRC11:33
jaosoriorshardy: that's what's messed up. In the minimal example, the heat-cfn logs show "authorization success", however, the deployment is still stuck, and when I try to do the curl, that keeps failing like before11:34
jaosoriormakes no sense11:34
jaosoriorand I have no idea what else to tryu11:34
shardyso, the curl call fails, but you don't see "AWS authentication failure." in the logs?11:34
jaosorioryeah11:35
shardyOk, so that's not consistent with your earlier data11:35
jaosoriorwhich is why  I'm so confused11:35
jaosoriorall I added was debug logs11:36
*** zoli|doctor is now known as zoli|wfh11:36
*** masco has quit IRC11:36
jaosoriorAnyway, I think my environment is messed up11:40
jaosoriorshardy: this is how it's failing in the gate, which is what I had initially been able to reproduce http://logs.openstack.org/37/394837/4/check/gate-tripleo-ci-centos-7-nonha-multinode/8539cdf/logs/var/log/heat/heat-api_cfn.txt.gz#_2016-11-10_15_19_58_11411:40
shardyjaosorior: I'd push a WIP patch to keystoneclient adding debug and recheck with a Depends-On perhaps11:41
shardyhttp://logs.openstack.org/37/394837/4/check/gate-tripleo-ci-centos-7-nonha-multinode/8539cdf/logs/var/log/keystone/keystone.txt.gz#_2016-11-10_14_48_20_57611:42
shardycould that be related?11:42
shardyDoesn't seem to be getting as far as actually calculating the signature?11:42
jaosoriorqnot sure11:43
jaosoriorthat error is quite common as far as I've seen'11:43
*** fzdarsky|afk is now known as fzdarsky|lunch11:48
openstackgerritJulie Pichon proposed openstack/puppet-tripleo: Persist the UI config file  https://review.openstack.org/38859211:51
openstackgerritvinay kumar muddu proposed openstack/diskimage-builder: Build wholedisk images capable of UEFI secure boot  https://review.openstack.org/39660211:51
*** rhallisey has joined #tripleo11:55
*** hjensas has quit IRC12:00
-openstackstatus- NOTICE: Our OpenStack CI systems are stuck and no new jobs are submitted. Please do not recheck - and do not approve changes until this is fixed.12:02
*** spredzy has quit IRC12:03
*** maticue has joined #tripleo12:07
*** ooolpbot has joined #tripleo12:10
ooolpbotURGENT TRIPLEO TASKS NEED ATTENTION12:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/164107612:10
openstackLaunchpad bug 1641076 in tripleo "CI: master job fail on undercloud install because of missing ceilometer-dbsync command" [Critical,Confirmed]12:10
*** ooolpbot has quit IRC12:10
*** lucasagomes is now known as lucas-hungry12:10
*** hjensas has joined #tripleo12:13
*** shardy is now known as shardy_lunch12:13
openstackgerritOliver Walsh proposed openstack/diskimage-builder: Include lsb_release on redhat/centos7 images  https://review.openstack.org/39660712:16
*** hjensas has quit IRC12:17
*** hjensas has joined #tripleo12:18
*** jkilpatr has joined #tripleo12:22
*** hjensas has quit IRC12:23
*** hjensas has joined #tripleo12:25
*** dprince has joined #tripleo12:26
*** dougbtv has quit IRC12:26
*** adarazs is now known as adarazs_lunch12:27
*** thrash|g0ne is now known as thrash12:30
*** milan has joined #tripleo12:31
*** jprovazn has joined #tripleo12:32
*** ccamacho has quit IRC12:37
*** hjensas has quit IRC12:38
*** hjensas has joined #tripleo12:39
*** hjensas has quit IRC12:39
*** hjensas has joined #tripleo12:39
*** lblanchard has joined #tripleo12:39
jaosoriorshardy_lunch: so the "reason" it'12:41
jaosoriorit's spitting out in the logs is12:41
jaosoriorauth fail reason: The request you have made requires authentication..12:41
jaosoriorshardy_lunch: so you might be right, that it doesn't contain tyhe token and it doesn't even get to the signature calculation12:42
*** fzdarsky|lunch is now known as fzdarsky12:44
*** hjensas has quit IRC12:45
*** jaosorior has quit IRC12:47
*** pkovar has joined #tripleo12:50
*** hjensas has joined #tripleo12:51
*** hjensas has quit IRC12:51
*** hjensas has joined #tripleo12:51
*** arxcruz has quit IRC12:51
openstackgerritPradeep Kilambi proposed openstack/puppet-tripleo: Add panko service support  https://review.openstack.org/39643512:56
openstackgerritPradeep Kilambi proposed openstack/puppet-tripleo: Fix barbican server name to not use aodh hiera  https://review.openstack.org/39643412:57
*** pabelanger has quit IRC12:57
*** pabelanger has joined #tripleo12:57
-openstackstatus- NOTICE: Our OpenStack CI system is coming back online again. Thanks for your patience.13:01
*** trown|outtypewww is now known as trown13:01
*** jayg|g0n3 is now known as jayg13:07
openstackgerritMerged openstack/tripleo-quickstart: Force the use of python version 2.7  https://review.openstack.org/36801313:07
*** ooolpbot has joined #tripleo13:10
ooolpbotURGENT TRIPLEO TASKS NEED ATTENTION13:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/164107613:10
openstackLaunchpad bug 1641076 in tripleo "CI: master job fail on undercloud install because of missing ceilometer-dbsync command" [Critical,Confirmed]13:10
*** ooolpbot has quit IRC13:10
pandawill a ssl + ipv6 deployment work ?13:14
*** adarazs_lunch is now known as adarazs13:19
openstackgerritBrent Eagles proposed openstack/tripleo-heat-templates: Add fail_mode workaround for connectivity issues on reboot  https://review.openstack.org/39628513:20
*** arxcruz has joined #tripleo13:22
*** arxcruz has quit IRC13:27
*** ecerquei has joined #tripleo13:27
*** arxcruz has joined #tripleo13:27
*** lucas-hungry is now known as lucasagomes13:29
*** phpcodemonkey has joined #tripleo13:31
florianfMy current deployment seems to have literally fallen asleep at AllNodesDeploySteps. It's calling heat.engine.scheduler._sleep every couple of seconds.13:31
*** hrybacki is now known as hrybacki|VDayRit13:31
florianfI wonder if it's ever going to wake up from that.13:31
openstackgerritBrent Eagles proposed openstack/puppet-tripleo: Call VF configuration from udev rules  https://review.openstack.org/39460413:32
*** rlandy has joined #tripleo13:34
openstackgerritBrent Eagles proposed openstack/os-net-config: Add support for enabling hotplug on interfaces  https://review.openstack.org/39466013:35
*** phpcodemonkey has quit IRC13:35
pandabeagles: it's friday for the deployments too.13:36
*** ecerquei has quit IRC13:36
*** ecerquei has joined #tripleo13:36
*** jpena is now known as jpena|lunch13:36
*** zaneb has joined #tripleo13:37
openstackgerritBrent Eagles proposed openstack/tripleo-heat-templates: Increase reserved memory for computes when enabling DVR  https://review.openstack.org/38259413:40
openstackgerritChris Jones proposed openstack/puppet-tripleo: Fix puppet lint failure  https://review.openstack.org/39663113:42
*** ecerquei has quit IRC13:42
*** zaneb has quit IRC13:42
Ng^^ will get puppet-tripleo CI moving again on newton. It's a straight backport of bnemec's fix from master13:43
NgI (and some others I suspect :) would very much appreciate it if a couple of cores could help land it :)13:43
*** rbowen has quit IRC13:44
Ngslagle: :)13:45
openstackgerritMerged openstack/tripleo-heat-templates: Fix race during major-upgrade-pacemaker step  https://review.openstack.org/39546013:51
*** cdearborn has joined #tripleo13:52
*** tzumainn has joined #tripleo13:53
*** tiswanso has joined #tripleo13:53
*** ecerquei has joined #tripleo13:53
*** ecerquei has quit IRC13:54
*** ecerquei has joined #tripleo13:54
*** amoralej is now known as amoralej|lunch13:54
*** ecerquei has quit IRC13:57
*** ealcaniz has quit IRC13:57
*** fultonj has joined #tripleo13:58
*** dprince has quit IRC14:00
*** dprince has joined #tripleo14:01
*** dciabrin has quit IRC14:02
jpichHere's a couple of patches under 3 lines that'd be helpful, if folks have a bit of time to review: https://review.openstack.org/#/c/393719/ https://review.openstack.org/#/c/388592/ (puppet-tripleo, instack-undercloud with one +2 already)14:03
*** zoli|wfh is now known as zoli|afk14:05
jtomasekflorianf: do we already have a bug or blueprint for using heat zaqar messages instead of polling?14:07
*** abregman|afk is now known as abregman14:07
florianfjtomasek: https://blueprints.launchpad.net/tripleo/+spec/ui-deployment-progress-websockets14:08
jtomasekflorianf: nice, thanks14:08
florianfjtomasek: not very detailed, but it's there...14:08
*** ooolpbot has joined #tripleo14:10
ooolpbotURGENT TRIPLEO TASKS NEED ATTENTION14:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/164107614:10
*** ooolpbot has quit IRC14:10
openstackLaunchpad bug 1641076 in tripleo "CI: master job fail on undercloud install because of missing ceilometer-dbsync command" [Critical,Confirmed]14:10
*** abregman is now known as abregman|afk14:12
openstackgerritAna Krivokapic proposed openstack/tripleo-ui: Replace direct reference of Mistral workflow with constant  https://review.openstack.org/39664314:12
*** rhefner has joined #tripleo14:13
*** jkilpatr_ has joined #tripleo14:14
*** jkilpatr has quit IRC14:16
*** shardy_lunch is now known as shardy14:18
*** amoralej|lunch is now known as amoralej14:21
*** ccamacho has joined #tripleo14:21
ccamachoHey Ng, around?14:21
ccamachoquick question, from the backported patch14:21
ccamachochecking http://logs.openstack.org/92/396092/2/check/gate-puppet-tripleo-puppet-lint/324ea78/console.html14:21
Ngccamacho: hey14:22
ccamachothe job its fetching 2016-11-11 13:25:18.923640 | Fetching https://git.openstack.org/openstack/puppet-openstack_spec_helper14:22
ccamachowithout the branch14:22
ccamachoso its getting master instead of newton, right?14:22
Ngccamacho: looks like it, yeah14:23
ccamacho2016-11-11 13:26:59.320683 | Using puppet-openstack_spec_helper 10.0.0 from https://git.openstack.org/openstack/puppet-openstack_spec_helper (at master@6b7aa95)14:23
ccamacho:P14:23
ccamachothats the actual issue then, not the backport14:23
*** Vijayendra has quit IRC14:24
Ngccamacho: interesting, so presumably I need to track down whatever owns the gate-puppet-tripleo-puppet-lint jenkins job, and fix that?14:27
ccamachoNg yeahp, Im trying to get it also but had no luck yet :)14:28
*** jcoufal has joined #tripleo14:28
openstackgerritTomas Sedovic proposed openstack/tripleo-docs: Add tripleo-validations user documentation  https://review.openstack.org/39619714:29
*** hrybacki|VDayRit is now known as hrybacki14:29
*** gbarros has joined #tripleo14:30
*** abehl has quit IRC14:31
Ngccamacho: would it be the Gemfile in puppet-tripleo? That specifies to grab puppet-openstack_spec_helper from git, with no specific branch14:31
ccamachoI think yes :)14:31
ccamachoNg but Im checking also other projects..14:32
ccamachoand they are the same..14:32
Ngpuppet-nova has :branch => 'stable/newton', in its newton branch14:32
Ngif nova does it, I'm comforatble pushing up a patch for tripleo to do it ;)14:32
trozetshardy: hi, for OOO when is the next design summit?14:33
ccamachoNg yeah! but I think they are also the same :(14:34
Ngccamacho: the same in what way?14:34
ccamachothey are not configuring the branch name14:34
shardytrozet: I believe after some debates we were granted space at the PTG in Atlanta14:35
openstackgerritAttila Darazs proposed openstack/tripleo-quickstart-extras: Add essential repo files  https://review.openstack.org/39665214:35
Ngccamacho: sorry, I don't understand. Their gemfile in newton is specifying the newton spec_helper. or do you mean something else?14:35
trozetshardy: ok do you have any links to that? or dates?14:35
shardytrozet: when Emilien gets back from PTO I expect more details will be circulated, I've not got any communication about it since handing over PTL responsibilities14:35
ccamachohttps://github.com/openstack/puppet-nova/blob/master/Gemfile  and https://github.com/openstack/puppet-tripleo/blob/master/Gemfile  i.e.14:36
trozetshardy: so Emilien is PTL now?14:36
shardytrozet: https://www.openstack.org/ptg/14:36
ccamachoNg^14:36
shardytrozet: yes, for ocata he is14:36
Ngccamacho: in master, yes, that seems right, since master is the default, but look at https://github.com/openstack/puppet-nova/blob/stable/newton/Gemfile14:36
ccamachoNg hey! your right, need more coffee14:37
ccamacho:)14:37
openstackgerritTomas Sedovic proposed openstack/tripleo-validations: Fix the check-network-gateway validation  https://review.openstack.org/39665414:37
ccamachothat will be the fix then..14:37
openstackgerritChris Jones proposed openstack/puppet-tripleo: Use stable/newton for puppet-openstack_spec_helper.  https://review.openstack.org/39665514:37
trozetshardy: ok cool.  I'm not going to make it to Brno, but ATL is 1 hour plane ride for me...so will probably see you at that one if you go14:37
ccamachoNg that would be the real bug :)14:37
ccamachoawesome14:37
openstackgerritTomas Sedovic proposed openstack/tripleo-validations: Fix the rabbitmq-limits validations  https://review.openstack.org/39665614:37
openstackgerritTomas Sedovic proposed openstack/tripleo-validations: Fix the pacemaker-status validation  https://review.openstack.org/39665714:38
shadowerwould appreciate reviews on the 3 patches above. They're just cherrypicks to tripleo-validations (all one-liners)14:40
*** d0ugal has joined #tripleo14:41
*** dsariel has quit IRC14:42
*** jpena|lunch is now known as jpena14:43
ccamachoNg im manually checking all puppet repos here https://etherpad.openstack.org/p/gemfile_check just in case you want to check them also :)14:44
Ngccamacho: I'll start at the bottom of the list :)14:45
ccamachoNg awesome :)14:45
openstackgerritMerged openstack/python-tripleoclient: Use correct region value  https://review.openstack.org/39451514:45
*** udesale has quit IRC14:46
openstackgerritMerged openstack/tripleo-heat-templates: Fix inconsistent Manila service naming  https://review.openstack.org/39631714:47
*** dsariel has joined #tripleo14:50
*** dciabrin has joined #tripleo14:51
mwhahahagfidente: whats up?14:53
bnemecshadower: This should be backported, right? https://review.openstack.org/#/c/396317/14:54
bnemecshardy: Same question^  Sorry shadower.14:55
* bnemec needs more caffeine14:56
Ngccamacho: so now the question is, do we just blindly submit patches to each of those projects? :)14:57
openstackgerritBrent Eagles proposed openstack/os-net-config: Add support for enabling hotplug on interfaces  https://review.openstack.org/39466014:57
Ngccamacho: if so, I'm happy to rattle through them14:57
ccamachoNg, the ones you tagged them as missing?14:58
Ngccamacho: yeah14:58
ccamachoThey are missing the branch tag?14:58
Ngyup14:58
ccamachoand they are using multiple branches?14:58
*** zoli|afk is now known as zoli|wfh14:58
ccamachoI know there are some only with master14:58
shardybnemec: yes I think so, I was just waiting for it to land14:58
Ngccamacho: oh that's a good point, I quickly scripted the check and didn't test for that. bah!14:59
shardybnemec: there's a small risk users could be impacted, but I think it's OK since we're changing all the environment files too14:59
shardyand the current syntax could be really confusing14:59
* shardy cherry-picks14:59
openstackgerritSteven Hardy proposed openstack/tripleo-heat-templates: Fix inconsistent Manila service naming  https://review.openstack.org/39666214:59
ccamachoNg, I think they should be fine if only using master, but in case using multiple branches the branch tag should be there15:00
bnemecshardy: Yeah, definitely agree.15:00
Ngccamacho: yeah, none of the MISSING ones have multiple branches, so looks like it's just puppet-tripleo :)15:00
openstackgerritGiulio Fidente proposed openstack/puppet-tripleo: Normalize civetweb binding address if IPv6  https://review.openstack.org/39578515:00
ccamachoNg awesome, maybe a snag when starting to use puppet-openstack_spec_helper :)15:01
Ng:)15:01
ccamachomissed puppet-tripleo15:01
openstackgerritArx Cruz proposed openstack/tripleo-heat-templates: Increasing neutron timeout for low memory usage  https://review.openstack.org/39036115:01
ccamachogood then15:01
ccamachoI8ll add https://etherpad.openstack.org/p/gemfile_check to the bug ticket15:01
gfidentemwhahaha, was trying to understand why previous patchset of https://review.openstack.org/395785 did not pass puppet 4 unit tests15:01
Ngccamacho: thank you very much for your insights and help!15:01
gfidentesocial, helped looking into it and it's passing now, if you could review that would be nice15:01
ccamachoNg no way! thanks to you for the help there :)15:01
mwhahahagfidente: sure15:02
openstackgerritGiulio Fidente proposed openstack/tripleo-heat-templates: Configure civetweb bind socket via puppet-tripleo  https://review.openstack.org/39056915:03
*** dprince has quit IRC15:03
*** florianf has quit IRC15:03
hrybackihey all -- having issues with mistral while trying to delete a node on the overcloud: I've hit the end of my debugging abilities and posted https://bugs.launchpad.net/mistral/+bug/1640933 can anyone give me some additional troubleshooting pointers? #openstack-mistral believes it is likely a #tripleo issue15:03
openstackLaunchpad bug 1640933 in Mistral "mistral raises 'Enviornment not found' for id of an existing stack while attempting to delete a node" [Undecided,New]15:03
*** dprince has joined #tripleo15:04
d0ugalhrybacki: Can you show me `mistral environment-list` ?15:07
*** florianf has joined #tripleo15:07
hrybackid0ugal: https://paste.fedoraproject.org/477940/47887687/15:07
d0ugalhrybacki: interesting - the environment seems to be there.15:08
*** ebalduf has joined #tripleo15:09
*** dmacpher has quit IRC15:09
hrybackiindeed strange. env details for the overcloud: https://paste.fedoraproject.org/477942/76947147/15:09
d0ugalhrybacki: Can you show me the delete command you used? I don't see that in the bug15:09
d0ugalhrybacki: did you pass "--stack overcloud" ?15:09
hrybackiyou know I didn't. I passed the stack id of the overcloud15:09
* hrybacki fetches commadn15:09
hrybackiopenstack overcloud node delete --debug --stack 06b0022a-7819-41c3-a5f9-f097a880e905 --templates -e /usr/share/openstack-tripleo-heat-templates/environments/network-isolation.yaml -e /usr/share/openstack-tripleo-heat-templates/environments/net-single-nic-with-vlans.yaml -e /home/stack/network-environment.yaml c8167eae-3695-439c-953f-42cd3df89d9315:10
*** ooolpbot has joined #tripleo15:10
ooolpbotURGENT TRIPLEO TASKS NEED ATTENTION15:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/164107615:10
openstackLaunchpad bug 1641076 in tripleo "CI: master job fail on undercloud install because of missing ceilometer-dbsync command" [Critical,Confirmed]15:10
*** ooolpbot has quit IRC15:10
hrybackiit's in the bug, just sort of hidden by the other stuff15:10
d0ugalhrybacki: aha, I think it assumes the stack name. I don't know if the ID would work.15:10
hrybackihrm, interesting15:10
d0ugalhrybacki: I have only tested this command with the name.15:10
hrybackid0ugal: testing now w/ 'overcloud'15:12
*** milan has quit IRC15:12
d0ugalhrybacki: cool, thanks. let me know how you get on15:13
d0ugalhrybacki: FWIW, it will actually default to 'overcloud' soon. https://review.openstack.org/#/c/395522/15:13
d0ugal(well, soon = in Ocata)15:13
hrybackid0ugal: that's it.15:13
d0ugalhrybacki: Great. So I guess we need to update your bug to give people a more sane error.15:14
hrybackiack, updating now15:14
rook-outhey shardy not sure the proper order of operations here but I added boot_option to ironic-inspector for puppet to clear out this bug https://bugs.launchpad.net/tripleo/+bug/1639858, here is my patch https://review.openstack.org/#/c/396667/ -- should i open a new bug against puppet-ironic to expose this option?15:16
openstackLaunchpad bug 1639858 in tripleo "[RFE] Enable by default boot_mode in inspector.conf" [Undecided,New]15:16
rook-outOnce my patch merges, we can close out the bug i reference ^15:17
d0ugalhrybacki: Thanks. Should be fairly easy to fix, I'll try on Monday.15:17
pandaderekh: (tripleo-ci) what does this call do ? curl http://$MIRRORSERVER/cgi-bin/upload.cgi  -F "repohash=$TRUNKREPOUSED" -F "$JOB_NAME=SUCCESS"15:18
hrybackid0ugal: added a comment to the bug. Would you like me to adjust the title or is it fine?15:19
pandaderekh: updates status page ?15:19
*** yamahata has joined #tripleo15:19
derekhpanda: At the end of the successfull periodic job it tells the mirror server that the jab names $JOB_NAME passed15:19
d0ugalhrybacki: I'll do a quick update15:20
pandaderekh: and what does the mirror server do in response ?15:20
derekhpanda: then on the mirror server there is a script that promotes the current repository if all the jobs passed15:20
dtantsurrook-out, reviewed. bug is only needed if you want it backported for some reason15:20
derekhpanda: here is the scipt its calling http://git.openstack.org/cgit/openstack-infra/tripleo-ci/tree/scripts/mirror-server/upload.cgi15:21
hrybackid0ugal: okay, thanks! /me is happy to have found the root cause15:21
rook-outdtantsur: ok15:21
derekhpanda: and here is the script that us run by the cron job http://git.openstack.org/cgit/openstack-infra/tripleo-ci/tree/scripts/mirror-server/promote.sh15:21
*** abregman|afk has quit IRC15:22
dtantsurrook-out, as to https://bugs.launchpad.net/tripleo/+bug/1639858, you'll also need an instack-undercloud patch to change the value15:22
openstackLaunchpad bug 1639858 in tripleo "[RFE] Enable by default boot_mode in inspector.conf" [Undecided,New]15:22
derekhpanda: the cron job is defined at the bottom of this file http://git.openstack.org/cgit/openstack-infra/tripleo-ci/tree/scripts/mirror-server/mirror-server.pp#n4615:22
openstackgerritGiulio Fidente proposed openstack/puppet-tripleo: Normalize civetweb binding address if IPv6  https://review.openstack.org/39578515:22
*** dougbtv has joined #tripleo15:23
pandaderekh: I understand, thanks.15:24
gfidentemwhahaha, updated, there is this guy which depends on https://review.openstack.org/#/c/390569/15:26
gfidente*depends on it15:26
mwhahahagfidente: ok15:26
*** liverpooler has joined #tripleo15:27
*** dprince has quit IRC15:27
gfidentemwhahaha, thanks15:27
ccamachogfidente mwhahaha mind to check when having some cycles https://review.openstack.org/#/c/396655/ ?? Its related to add stable/newton to the gemfile in puppet-tripleo (stable/newton) currently blocking backports.15:34
ccamachoA 1 liner fix15:34
ccamachothanks!15:35
hrybackiadarazs: jpena bkero dmsimard panda trown weshay please take a few minutes and add your thoughts to the rdo-infra retrospective board before the meeting: https://trello.com/b/0VFswmht/rdo-infra-retrospective thanks!15:35
dmsimardhrybacki: when's that meeting ?15:35
hrybackidmsimard: in ~55 minutes15:36
dmsimardhrybacki: holiday in Canada today :(15:36
trownhrybacki: wrong channel?15:37
trownhrybacki: fine here, but rdo-infra seems like a #rdo topic :)15:37
hrybackitrown: should have hit up #rdo yes, wrong channel :P15:37
hrybackisorry for spam #tripleo !15:38
*** liverpooler has quit IRC15:40
hrybackidmsimard: no worries!15:40
openstackgerritMerged openstack/puppet-tripleo: Ensure keepalived is restarted when necessary.  https://review.openstack.org/39505315:43
*** numans has quit IRC15:45
*** chandankumar has quit IRC15:45
Ngccamacho: jenkins is back and green for https://review.openstack.org/#/c/396655/1 :)15:45
*** fragatina has quit IRC15:45
ccamachoNg done :)15:45
Ng\o/15:45
*** fragatina has joined #tripleo15:46
slaglepradk: hi, could you work on a patch for tripleo-docs about the enable_telemetry flip flop we had to do for newton?15:50
slaglepradk: and the impact, etc15:50
pradkslagle, sure15:50
slagleshardy: pradk : fyi, there was some concern from the stable release team about these patches, and it's holding up our newton release15:50
slaglehttps://review.openstack.org/#/c/391799/15:50
slaglesee the comments on the patch from tony15:50
*** jprovazn has quit IRC15:51
pradkslagle, yea reading now.. so what we need is this behavior documented on what the default is and how to switch if needed?15:52
*** iranzo has quit IRC15:52
slaglepradk: yes, and particularly if they happened to upgrade to the first newton release, they got the new default (false), and now we're switching that back to true15:53
beaglesslagle: dprince: dsneddon: etc. I've added you as reviewers to a couple of patches aimed at https://bugs.launchpad.net/tripleo/+bug/1640812 ...15:53
openstackLaunchpad bug 1640812 in tripleo "Network connectivity lost on node reboot" [Critical,In progress] - Assigned to Brent Eagles (beagles)15:53
pradkslagle, understood, i'll get a docs patch out today15:53
slaglethanks!15:54
beaglesit's kind of a weird (yet monstrous) issue and the workaround may seem a bit odd... I'm concerned about corner cases, unexpected consequences etc.15:54
shardyslagle: ack, thanks for the update15:55
shardyhopefully some docs will prove sufficient to unblock things15:56
beaglesI'm enumerating different configuration types, looking for problems with this approach, but I know I'll miss some15:56
*** dsariel has quit IRC16:01
*** ebarrera has quit IRC16:01
*** rcernin has quit IRC16:05
*** zaneb has joined #tripleo16:05
yolandahi, can i get some reviews on https://review.openstack.org/394426 ? to support whole disk images in tripleo client16:06
ansiwenmwhahaha: what do you mean with release notes? a comment which says which release is required for that option?16:07
trownyolanda: added myself to that review, I will take a look this afternoon16:07
mwhahahaansiwen: a release note using reno so people who read our release notes will see the added option, http://docs.openstack.org/developer/reno/16:08
yolandathanks16:08
*** ooolpbot has joined #tripleo16:10
ooolpbotURGENT TRIPLEO TASKS NEED ATTENTION16:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/164107616:10
openstackLaunchpad bug 1641076 in tripleo "CI: master job fail on undercloud install because of missing ceilometer-dbsync command" [Critical,Confirmed]16:10
*** ooolpbot has quit IRC16:10
shardyjistr: Hey can we close https://bugs.launchpad.net/tripleo/+bug/1635606 or are more patches needed?16:10
openstackLaunchpad bug 1635606 in tripleo "Lost ability to deploy Glance backed by NFS" [High,In progress] - Assigned to Jiří Stránský (jistr)16:10
shardyI started setting the topic for patches associated with bugs targetted at ocata-116:11
jistrshardy: ah yes, thanks for the reminder. I set it to fix-commited16:11
shardyhttps://review.openstack.org/#/q/topic:tripleo/ocata-116:11
shardyover the next few days we can land as many of the bugfixes as we can, then I'll defer any non-critical issues on Tuesday next week16:12
shardyjistr: thanks!16:12
*** paramite has quit IRC16:13
*** fragatina has quit IRC16:16
shardyslagle: hey can we declare https://bugs.launchpad.net/tripleo/+bug/1637204 complete for ocata-1 ?16:16
openstackLaunchpad bug 1637204 in tripleo "Parameter 'CephStorageCount' was not found in existing stack when using composable roles" [High,In progress] - Assigned to James Slagle (james-slagle)16:16
openstackgerritArx Cruz proposed openstack/tripleo-quickstart: Configuring vcpus for undercloud, controller and compute nodes  https://review.openstack.org/39659316:17
*** tremble has quit IRC16:18
*** ramishra has quit IRC16:18
*** ramishra has joined #tripleo16:20
*** bnemec is now known as beekneemech16:22
*** ealcaniz has joined #tripleo16:22
*** ohamada has quit IRC16:25
slagleshardy: i declare it16:27
openstackgerritMerged openstack/tripleo-validations: Fix the rabbitmq-limits validations  https://review.openstack.org/39665616:27
shardyslagle: ack, thanks!16:27
*** mcornea has quit IRC16:27
openstackgerritMerged openstack/tripleo-validations: Fix the pacemaker-status validation  https://review.openstack.org/39665716:28
openstackgerritMerged openstack/tripleo-validations: Fix the check-network-gateway validation  https://review.openstack.org/39665416:28
*** ealcaniz has quit IRC16:28
*** abregman has joined #tripleo16:29
openstackgerritAttila Darazs proposed openstack/tripleo-quickstart: Revert "Force the use of python version 2.7"  https://review.openstack.org/39668916:30
openstackgerritPradeep Kilambi proposed openstack/tripleo-docs: Explicitly document enable_telemetry defaults in upgrade docs  https://review.openstack.org/39669016:30
openstackgerritMerged openstack/tripleo-validations: undercloud-tokenflush: changed group to pre-introspection  https://review.openstack.org/39621216:31
*** arxcruz has quit IRC16:31
openstackgerritMerged openstack/tripleo-quickstart: Revert "Force the use of python version 2.7"  https://review.openstack.org/39668916:34
openstackgerritPradeep Kilambi proposed openstack/tripleo-heat-templates: Add panko api support to service templates  https://review.openstack.org/39643916:34
*** abregman is now known as abregman|nb16:35
openstackgerritTomas Sedovic proposed openstack/tripleo-docs: Add tripleo-validations user documentation  https://review.openstack.org/39619716:36
*** pkovar has quit IRC16:37
openstackgerritMerged openstack/puppet-tripleo: Use stable/newton for puppet-openstack_spec_helper.  https://review.openstack.org/39665516:38
*** jlinkes has quit IRC16:42
*** abregman|nb is now known as abregman16:43
openstackgerritAlejandro Andreu proposed openstack/tripleo-heat-templates: WIP: Create MidoNet services  https://review.openstack.org/39669216:43
*** Guest7605 has quit IRC16:43
openstackgerritDougal Matthews proposed openstack/python-tripleoclient: Format the nodes list in openstack overcloud delete node  https://review.openstack.org/39508316:45
openstackgerritAttila Darazs proposed openstack/tripleo-quickstart: WIP: shell script to merge roles into extras  https://review.openstack.org/39669416:51
*** hewbrocca is now known as hewbrocca_afk16:53
openstackgerritMerged openstack/instack-undercloud: Remove instack-create-overcloudrc  https://review.openstack.org/38882316:55
*** pkovar has joined #tripleo16:56
*** david-lyle has quit IRC16:58
*** lmiccini has quit IRC16:58
*** lblanchard has quit IRC16:59
*** lmiccini has joined #tripleo16:59
*** panda is now known as panda|bbl17:00
*** rajinir has joined #tripleo17:01
*** abregman is now known as abregman|afk17:03
*** jprovazn has joined #tripleo17:07
*** gbarros has quit IRC17:08
openstackgerritOliver Walsh proposed openstack/diskimage-builder: In disk-image-create, append to INSTALL_PACKAGES instead of clobbering. This allows -p <packagelist> to be used multiple times.  https://review.openstack.org/39670217:09
*** ooolpbot has joined #tripleo17:10
ooolpbotURGENT TRIPLEO TASKS NEED ATTENTION17:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/164107617:10
*** ooolpbot has quit IRC17:10
openstackLaunchpad bug 1641076 in tripleo "CI: master job fail on undercloud install because of missing ceilometer-dbsync command" [Critical,Confirmed]17:10
openstackgerritOliver Walsh proposed openstack/diskimage-builder: In disk-image-create, append to INSTALL_PACKAGES instead of clobbering.  https://review.openstack.org/39670217:12
*** dbecker has quit IRC17:12
*** jkilpatr_ has quit IRC17:12
openstackgerritMerged openstack/instack-undercloud: Open firewall port for the TripleO UI  https://review.openstack.org/39371917:13
*** abregman|afk is now known as abregman|nb17:14
*** gbarros has joined #tripleo17:15
*** jpich has quit IRC17:16
openstackgerritMerged openstack/os-net-config: Add route_options parameter  https://review.openstack.org/38891617:16
mariosshardy: hey around?17:18
shardymarios: hey, yup!17:18
*** dprince has joined #tripleo17:19
*** ecerquei has joined #tripleo17:19
mariosshardy: hi... are we still backporting things for liberty? we have an ovs workaround which we backported to mitaka already but been told we need it in liberty too (and also kilo but ignoring that for now)17:19
mariosshardy: https://bugs.launchpad.net/tripleo/+bug/1636748 and https://bugs.launchpad.net/tripleo/+bug/1635205 together (2 patches) make the fix17:19
openstackLaunchpad bug 1636748 in tripleo "Mitaka to Newton upgrade ovs upgrade check failed on ovs 2.5" [Undecided,Fix released] - Assigned to Marios Andreou (marios-b)17:19
openstackLaunchpad bug 1635205 in tripleo "Mitaka to Newton upgrade - ovs 2.4 to 2.5 causes nodes to lose ovs interfaces" [Critical,Fix released] - Assigned to Marios Andreou (marios-b)17:20
mariosshardy: i have added mitaka-backport-potential to those and done the backport and it landed. is it just liberty-backport-potential? I ask because the tag wasn't found when  i just tried to add it17:20
openstackgerritJulie Pichon proposed openstack/instack-undercloud: Open firewall port for the TripleO UI  https://review.openstack.org/39670617:20
shardymarios: we can definitely backport them to newton/mitaka, but liberty goes EOL next week17:21
shardySo you may decide not to backport there17:22
shardyhttps://releases.openstack.org/17:22
mariosshardy: ok thanks17:22
*** michapma_alt has quit IRC17:23
*** dtantsur is now known as dtantsur|afk17:24
openstackgerritJiri Tomasek proposed openstack/tripleo-specs: GUI Deployment Configuration improvements  https://review.openstack.org/39336517:24
dprincea quick review here would help me: https://review.openstack.org/#/c/396175/17:26
dprincepassing CI ^^^17:26
*** jkilpatr_ has joined #tripleo17:26
*** b00tcat` has quit IRC17:30
trowndprince: so currently if role_data is null, we get data=null and the expression breaks somehow, but your updated wrapping in an extra dict we get data={role_data: null} and it works?17:32
trownoh or does it just break immediately when trying to pass null as "data" but passing a dict with the null as a value works?17:34
openstackgerritJiri Stransky proposed openstack/tripleo-common: Only use parameter_defaults for scaling down  https://review.openstack.org/39671217:36
akrivokajtomasek: still here?17:39
*** derekh has quit IRC17:42
openstackgerritJiri Stransky proposed openstack/tripleo-common: Only use parameter_defaults for scaling down  https://review.openstack.org/39671217:43
openstackgerritSteven Hardy proposed openstack/tripleo-heat-templates: WIP prototyping composable upgrades with Heat+Ansible  https://review.openstack.org/39344817:43
*** shardy has quit IRC17:43
openstackgerritSteven Hardy proposed openstack/tripleo-heat-templates: WIP prototyping composable upgrades with Heat+Ansible  https://review.openstack.org/39344817:44
*** zaneb has quit IRC17:47
openstackgerritBen Nemec proposed openstack-infra/tripleo-ci: POC: Use tripleo-repos in tripleo.sh  https://review.openstack.org/39581317:48
*** lucasagomes is now known as lucas-afk17:48
*** zaneb has joined #tripleo17:49
*** sudipto has quit IRC17:51
*** sudipto_ has quit IRC17:51
*** yamahata has quit IRC17:55
*** Baggypants12000 has joined #tripleo17:55
*** d0ugal has quit IRC17:57
jtomasekakrivoka: now17:58
*** david-lyle has joined #tripleo17:58
akrivokajtomasek: so I'm trying to understand how the new plan import feature would differ from simple plan creation (what we have now). with the new plan import, the configuration of environments and parameters would be stored so the user does not have to manually set them? correct? is that the point?18:00
*** Goneri has joined #tripleo18:01
*** pkovar has quit IRC18:01
*** tiswanso has quit IRC18:01
*** tosky has quit IRC18:02
*** david-lyle has quit IRC18:03
jtomasekakrivoka: yes, correct. Import is a create, we just need to enhance plan creation so it consumes the mistral-environment.yaml (json)18:03
jtomasek(plan update needs to support this too)18:03
akrivokajtomasek: cool, thanks, just wanted to check my understanding was correct18:04
*** mhenkel has joined #tripleo18:04
jtomasek akrivoka: np18:04
akrivokajtomasek: now switch the computer off start pto-ing :)18:05
*** chandankumar has joined #tripleo18:05
*** athomas has quit IRC18:06
*** trown is now known as trown|lunch18:06
*** bana_k has joined #tripleo18:07
openstackgerritPierre Blanc proposed openstack/os-net-config: Add support for ethtool_opts option  https://review.openstack.org/39269718:08
*** fragatina has joined #tripleo18:10
*** ooolpbot has joined #tripleo18:10
ooolpbotURGENT TRIPLEO TASKS NEED ATTENTION18:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/164107618:10
*** ooolpbot has quit IRC18:10
openstackLaunchpad bug 1641076 in tripleo "CI: master job fail on undercloud install because of missing ceilometer-dbsync command" [Critical,Confirmed]18:10
*** rcernin has joined #tripleo18:11
*** rcernin has quit IRC18:12
*** d0ugal has joined #tripleo18:12
jtomasekakrivoka: one more review reply is required:)18:13
*** aufi_ has quit IRC18:15
*** tiswanso has joined #tripleo18:21
*** tiswanso has quit IRC18:22
*** tiswanso has joined #tripleo18:23
openstackgerritPradeep Kilambi proposed openstack/tripleo-docs: Explicitly document enable_telemetry defaults in upgrade docs  https://review.openstack.org/39669018:24
*** tiswanso has quit IRC18:24
*** tiswanso has joined #tripleo18:25
*** fragatina has quit IRC18:27
*** fragatina has joined #tripleo18:28
*** rhallisey has quit IRC18:30
*** rhallisey has joined #tripleo18:31
*** mgould is now known as mgould|afk18:32
*** jpena is now known as jpena|off18:33
*** rasca has quit IRC18:34
*** yamahata has joined #tripleo18:34
*** bana_k has quit IRC18:35
*** david-lyle has joined #tripleo18:36
*** pblaho has quit IRC18:38
openstackgerritBob Fournier proposed openstack/os-net-config: Stop dhclient in os-net-config if interface not set for DHCP  https://review.openstack.org/39644618:38
*** lmiccini has quit IRC18:42
*** yamahata has quit IRC18:44
*** yamahata has joined #tripleo18:45
slaglebeekneemech: wasn't telemetry disabled in the 5.0.0 release of i-u?18:47
slaglethat's what the docs are meant to address, people who are on 5.0.018:47
beekneemechslagle: But it's not possible to install that anymore.  The final newton release (presumably 5.0.something) flipped the default back.18:48
openstackgerritMarios Andreou proposed openstack/puppet-tripleo: Ensure keepalived is restarted when necessary.  https://review.openstack.org/39673118:49
beekneemechWe're basically documenting a fixed bug.18:49
slaglebeekneemech: oh so you're saying we don't need the last part of the comment?18:49
jristwould love one more core eye on this if possible https://review.openstack.org/#/c/391259/118:49
beekneemechslagle: I'm not sure we need any of that comment at all.  The only way you could run into it is if you upgraded from Mitaka to Newton rc2, which is not a supported release.18:51
*** bana_k has joined #tripleo18:51
beekneemechUnless I'm misunderstanding the intent of release candidates.18:51
slaglebeekneemech: we did a real release for newton already, 5.0.018:52
beekneemechI guess there's a CD argument here, but if you're CD'ing you've already hit the bug and gotten passed it.18:52
slagleenable_telemetry=false was in that release18:53
beekneemechslagle: And how would you install that today?18:53
beekneemechPick a random old dlrn repo?18:53
slaglenot random, use rdo newton18:53
beekneemechIs RDO Newton not using the actual released version of TripleO/18:53
slagleor use the release tarball, although i agree that seems unlikely to have been done18:53
beekneemechI guess maybe the problem here is we released instack-undercloud as part of an RC, but didn't tag the instack-undercloud version as RC?18:54
beekneemechSo there's a mismatch between the package version and the "TripleO" version.18:55
slaglewasn't 5.0.0 a non-rc release of instack-undercloud?18:55
beekneemechslagle: I don't know.  We released newton rc3 on Oct. 18: https://github.com/openstack/releases/commit/674605c37c495fd8f9090ba6f0a800e1eee8c27018:57
beekneemechWe merged the patch to change the default on Oct. 14: https://github.com/openstack/instack-undercloud/commit/8a738272c7f63ad7e6e00f0836c2d1793ea4f12518:57
beekneemechOh wait, but we had branched.18:57
beekneemechCrap, let me check stable/newton.18:58
*** Goneri has quit IRC18:58
slaglehttps://review.openstack.org/#/c/388799/18:59
*** jkilpatr_ has quit IRC18:59
beekneemechAh hell, that didn't merge until Oct. 31, which is after the final release was cut.18:59
openstackgerritJohn Trowbridge proposed openstack/tripleo-quickstart: add doc regarding how to console into the libvirt guests  https://review.openstack.org/39298318:59
*** trown|lunch is now known as trown19:00
*** jkilpatr has joined #tripleo19:01
beekneemechslagle: Okay, I +2'd the docs patch.  It looks like we did release with the changed default. :-/19:01
slagleok19:01
slaglei guess we should maybe hold off on approving the docs patch until the new release is actually done19:02
*** d0ugal has quit IRC19:02
beekneemechslagle: Did we not do another release last week?19:03
openstackgerritTim Rozet proposed openstack/tripleo-heat-templates: Fixes missing OVS Firewall config with OpenDaylight  https://review.openstack.org/39673419:03
slaglebeekneemech: no. i requested a new one this week19:03
beekneemechslagle: Ah, okay.19:03
* beekneemech should just stay out of release-related stuff19:03
slaglethis is one of the patches the release team wanted us to address19:03
slagleor would like us to anyway19:04
slaglewe dont have the stable-follows-policy tag, but we should be aiming for it19:04
beekneemech+119:04
openstackgerritMerged openstack/tripleo-quickstart: add doc regarding how to console into the libvirt guests  https://review.openstack.org/39298319:05
*** eggmaster has quit IRC19:05
openstackgerritMerged openstack/tripleo-ui: Replace direct reference of Mistral workflow with constant  https://review.openstack.org/39664319:06
*** pcaruana has quit IRC19:07
*** lblanchard has joined #tripleo19:07
openstackgerritBen Nemec proposed openstack/tripleo-docs: Add Newton repo details  https://review.openstack.org/39673619:08
openstackgerritwes hayutin proposed openstack/tripleo-quickstart: [WIP] config for containerized-compute  https://review.openstack.org/39334819:09
*** ooolpbot has joined #tripleo19:10
ooolpbotURGENT TRIPLEO TASKS NEED ATTENTION19:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/164107619:10
openstackLaunchpad bug 1641076 in tripleo "CI: master job fail on undercloud install because of missing ceilometer-dbsync command" [Critical,Confirmed]19:10
*** ooolpbot has quit IRC19:10
trownshould we remove alert tag from ^19:11
trownpatch to fix should merge soon19:11
trownor do we wait until we see promote pass?19:12
openstackgerritAndy Smith proposed openstack/tripleo-specs: Enable Oslo.Messaging Dual Backends  https://review.openstack.org/39674019:12
openstackgerritwes hayutin proposed openstack/tripleo-quickstart: [WIP] config for containerized-compute  https://review.openstack.org/39334819:18
openstackgerritMerged openstack/puppet-tripleo: Enable TLS in the internal network for Cinder API  https://review.openstack.org/39209019:19
openstackgerritMerged openstack/tripleo-heat-templates: Handle null role_data in services  https://review.openstack.org/39617519:20
*** zoli|wfh is now known as zoli|gone19:21
openstackgerritMerged openstack/puppet-tripleo: Improve failed mysql node removal time in HA deploys.  https://review.openstack.org/39609219:23
openstackgerritMerged openstack/puppet-tripleo: Normalize civetweb binding address if IPv6  https://review.openstack.org/39578519:23
*** zaneb has quit IRC19:24
*** fzdarsky is now known as fzdarsky|afk19:25
*** zaneb has joined #tripleo19:26
*** hjensas has quit IRC19:28
openstackgerritBrent Eagles proposed openstack/tripleo-heat-templates: Add fail_mode workaround for connectivity issues on reboot  https://review.openstack.org/39628519:30
dtrainoranyone got a sec to talk to me about NeutronNetworkType, NeutronTunnelTypes, how they relate, if they need to be of the same type/value, etc?19:33
*** akrivoka has quit IRC19:34
*** gbarros has quit IRC19:37
*** hjensas has joined #tripleo19:37
*** gfidente has quit IRC19:41
beaglesdtrainor: the NeutronNetworkType (could be better named I admit) specifies types of tenant networks you want to allow neutron to create19:43
*** rhallisey has quit IRC19:43
panda|bbltrown: I believe the alert is needed also so people know they may have problems with patches, not only to attract fixers.19:43
beaglesdtrainor: the tunnel types are basically gre and vxlan, and they are also tenant network types19:44
dtrainorunderstood.  can they mix and match or do they need to be the same for both Network and Tunnel types?19:44
dtrainor(does it matter?)19:44
trownpanda|bbl: k, in this case though it only affects promote... I just worry if the alert just becomes spam that is ignored19:45
beaglesdtrainor: if you specify vxlan and/or gre, they must also appear in the NeutronNetworkType - tunnel types that aren't in the tenant networks is kind of meaningless and should probably error out19:45
dtrainorok thanks.  figured as much but wanted confirmation.  sorry if i'm not too knowledgeable on the subject.19:45
*** rajinir has quit IRC19:46
dtrainori saw an opportunity for a validation19:49
*** rajinir has joined #tripleo19:49
beaglesdtrainor: tbh, considering how we are using it, I'm not sure it makes sense to have both parameters19:50
*** gbarros has joined #tripleo19:50
dtrainoroh yeah?19:51
beaglesdtrainor: in any case all of those command line arguments are deprecated so that statement is kind of moot :)19:52
dtrainoractually you're right :) I remember seeing that.19:53
*** Goneri has joined #tripleo19:53
*** jprovazn has quit IRC19:53
*** jprovazn has joined #tripleo19:55
*** tbonds has quit IRC20:06
*** bana_k has quit IRC20:07
beekneemechpanda|bbl: But people aren't going to have a problem with patches because it only affects an un-promoted version of ceilometer.20:09
beekneemechSince the patch is approved, I think we should drop the alert.20:09
*** chandankumar has quit IRC20:10
beekneemechI'm not entirely sure we should alert on master issues anyway.  Critical is fine, but those aren't blockers.20:10
*** ooolpbot has joined #tripleo20:10
ooolpbotURGENT TRIPLEO TASKS NEED ATTENTION20:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/164107620:10
openstackLaunchpad bug 1641076 in tripleo "CI: master job fail on undercloud install because of missing ceilometer-dbsync command" [Critical,Confirmed] - Assigned to Alfredo Moralejo (amoralej)20:10
*** ooolpbot has quit IRC20:10
openstackgerritMike Burns proposed openstack/tripleo-image-elements: Include lsb_release in the hosts element since it's used there.  https://review.openstack.org/39675920:11
openstackgerritwes hayutin proposed openstack/tripleo-quickstart: [WIP] config for containerized-compute  https://review.openstack.org/39334820:12
openstackgerritDan Prince proposed openstack/tripleo-puppet-elements: Disable old hiera.yaml o-a-c script  https://review.openstack.org/39261520:13
*** amoralej is now known as amoralej|off20:13
*** panda|bbl is now known as panda20:20
*** bana_k has joined #tripleo20:20
openstackgerritMike Burns proposed openstack/tripleo-image-elements: Include lsb_release in the hosts element since it's used there.  https://review.openstack.org/39675920:21
openstackgerritMerged openstack/tripleo-heat-templates: Increasing neutron timeout for low memory usage  https://review.openstack.org/39036120:21
*** bana_k has quit IRC20:25
openstackgerritDan Sneddon proposed openstack/os-net-config: Stop dhclient in os-net-config if interface not set for DHCP  https://review.openstack.org/39644620:25
pandaI need to take a look at the promotion diagram again ? why this package was only blocking promotion and e.g. redis was blocking the gates too ?20:25
mburnedbeekneemech: thanks for the quick review...20:26
beekneemechmburned: So I just realized that the yum-minimal example I linked to has a bug.20:28
beekneemechIt's not referring to the pkg-map name correctly.20:29
mburnedno worries...20:29
mburnednot sure if this is the end fix or if it's https://review.openstack.org/#/c/396607/1 (though that will have the same issues, i expect...)20:29
*** zaneb has quit IRC20:30
openstackgerritBen Nemec proposed openstack/diskimage-builder: Install lsb package by map name instead of package name  https://review.openstack.org/39676220:31
openstackgerritMike Burns proposed openstack/tripleo-image-elements: Include lsb_release in the hosts element since it's used there.  https://review.openstack.org/39675920:31
beekneemechmburned: I prefer yours.  The element that uses the package should list it as a dep.20:32
mburnedbeekneemech: yeah, i sorta agree too...20:32
beekneemechThe funny thing is I just noticed this today because I ran a CI job that accidentally didn't include ceph and it failed on an lsb-release call. :-)20:32
mburnedyeah, that's what caused this too20:34
mburnedci escalations...20:34
*** abregman|nb is now known as abregman|afk20:37
*** zaneb has joined #tripleo20:39
trownpanda: it is because ceilometer removed the deprecated ceilometer-dbsync command... so even though fix is in puppet-ceilometer (which we follow master of in all CI), the break is in ceilometer itself which we have pinned20:39
*** akrzos has quit IRC20:40
trownpanda: I am not familiar with the redis issue, so I dont have a reverse explanation there20:40
trownpanda: was the issue in puppet-redis?20:40
pandatrown: no20:41
trownoh right redis is in deps repo20:41
trownwe can't pin deps repo, it is not a static repo like dlrn repos20:41
*** dsariel has joined #tripleo20:41
*** akrzos has joined #tripleo20:41
openstackgerritBen Nemec proposed openstack-infra/tripleo-ci: Always use cached images in updates job  https://review.openstack.org/38671320:42
openstackgerritJohn Trowbridge proposed openstack/tripleo-heat-templates: Increasing neutron timeout for low memory usage  https://review.openstack.org/39676820:42
*** ccamacho has quit IRC20:44
pandatrown: why was ceilometer pinned ? or is pinning part of the promotion ? the command was deprecated a long time ago, do we need some tool that gathers all deprecation warnings so they can be addressed before it's too late ?20:44
trownpanda: everything in dlrn is pinned that does not have a tripleo gate (all the things in current-tripleo repo list are "unpinned")20:45
*** akrzos has quit IRC20:47
pandatrown: https://media.tenor.co/images/5fc0bd75396977ab93e0b3405edb5c4e/raw20:54
pandatrown: ok, thanks.20:55
trownpanda: is that zach galafanakis? what is that frome :)20:55
*** florianf has quit IRC20:55
trownoh hangover ;)20:56
trownhit me on my beeper20:56
pandatrown: https://www.youtube.com/watch?v=l2qZY5WFOYg&t=1m22s20:57
trownthose equations have integrals, which make no sense in discrete math used for blackjack calculations... just saying21:00
*** ccamacho has joined #tripleo21:00
*** maticue has quit IRC21:03
*** akrzos has joined #tripleo21:04
openstackgerritMerged openstack/tripleo-heat-templates: Enable internal TLS for Cinder API  https://review.openstack.org/39209521:04
pandatrown: Fermat's theorem was proved with elliptic curves.21:04
openstackgerritBrent Eagles proposed openstack/os-net-config: Add support for enabling hotplug on interfaces  https://review.openstack.org/39466021:05
trownpanda: blackjack math is pretty simple counting though :), but props on Fermat's last theorem knowledge21:06
*** dprince has quit IRC21:08
pandanow if only I could get this netiso ipv6 right.... :)21:09
*** akrzos has quit IRC21:09
*** florianf has joined #tripleo21:10
pandaI'm getting this error Could not fetch contents for file:///tmp/tripleoclient-WqEngI/network/ports/storage_v6.yaml. The netiso template uses relative paths, but I'm not sure if I should move it or specify a different base dir for the template (if possible)21:17
pandaMaybe I should read how "deploy command" code works21:17
*** akrzos has joined #tripleo21:18
openstackgerritMerged openstack/tripleo-heat-templates: Neutron L3 service cleanups for hiera json hook  https://review.openstack.org/39326221:19
pandaalso, this is really difficult to debug since if I do something wrong is just hangs. No errors. only timeouts ...21:20
mburnedbeekneemech: anything else needed on 396759 ?21:29
mburnedor can i cherry-pick to newton?21:29
trownmburned: just have to wait for it to pass CI, then we can merge and cherry-pick21:30
mburnedor is there another pipeline that runs...21:30
*** jprovazn has quit IRC21:30
trownah, ya there are more jobs that run21:30
mburnedack21:31
*** jayg is now known as jayg|g0n321:34
*** jkilpatr has quit IRC21:39
openstackgerritMerged openstack/puppet-tripleo: Fix puppet lint failure  https://review.openstack.org/39663121:42
openstackgerritMerged openstack/os-net-config: Add support for ethtool_opts option  https://review.openstack.org/39269721:42
*** florianf has quit IRC21:43
*** akrzos has quit IRC21:43
*** akrzos has joined #tripleo21:47
*** fultonj has quit IRC22:01
*** jcoufal has quit IRC22:07
*** bana_k has joined #tripleo22:08
*** cdearborn has quit IRC22:16
*** jkilpatr has joined #tripleo22:21
*** trown is now known as trown|outtypewww22:22
*** lblanchard has quit IRC22:30
*** akrzos has quit IRC22:35
*** akrzos has joined #tripleo22:36
*** dougbtv has quit IRC22:36
*** myoung is now known as myoung|afk22:38
*** mhenkel has quit IRC22:39
*** mhenkel has joined #tripleo22:49
gbarros#tripleo, looking for the actual executed cmdline of a command puppet is running during a deploy on a compute, where would I find that?23:06
beekneemechgbarros: You might want to try puppet-openstack for that question.  It's not really specific to tripleo and there are more puppet experts there.23:10
gbarrosty23:10
*** zaneb has quit IRC23:15
*** zaneb has joined #tripleo23:19
*** pradk has quit IRC23:19
openstackgerritBen Nemec proposed openstack-infra/tripleo-ci: Always use cached images in updates job  https://review.openstack.org/38671323:20
*** limao has joined #tripleo23:29
*** limao_ has joined #tripleo23:30
*** limao has quit IRC23:33
*** ccamacho has quit IRC23:40
*** panda is now known as panda|Zz23:42
*** zaneb has quit IRC23:52
*** pmannidi has joined #tripleo23:56

Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!