| *** saneax is now known as saneax_AFK | 00:17 | |
| *** chlong has joined #tripleo | 00:19 | |
| openstackgerrit | Ian Wienand proposed openstack/diskimage-builder: Do not remove sudo in ironic-agent https://review.openstack.org/308747 | 00:41 |
|---|---|---|
| openstackgerrit | Ian Wienand proposed openstack/diskimage-builder: Add 1.16.0 releasenotes https://review.openstack.org/316532 | 01:20 |
| *** myoung|lunch has joined #tripleo | 01:22 | |
| *** rwsu has quit IRC | 01:27 | |
| *** bswartz has quit IRC | 01:28 | |
| openstackgerrit | greghaynes proposed openstack/diskimage-builder: Properly fail/trap in eval_run_d https://review.openstack.org/259734 | 01:35 |
| openstackgerrit | greghaynes proposed openstack/diskimage-builder: Properly fail/trap in eval_run_d https://review.openstack.org/259734 | 01:37 |
| *** rwsu has joined #tripleo | 01:39 | |
| openstackgerrit | Merged openstack/diskimage-builder: Do not remove sudo in ironic-agent https://review.openstack.org/308747 | 01:41 |
| *** chlong has quit IRC | 01:56 | |
| *** myoung|lunch has quit IRC | 02:03 | |
| *** mbound has joined #tripleo | 02:09 | |
| *** shivrao has quit IRC | 02:10 | |
| *** mbound has quit IRC | 02:14 | |
| *** bswartz has joined #tripleo | 02:33 | |
| *** ccamacho has quit IRC | 02:43 | |
| *** ccamacho has joined #tripleo | 02:45 | |
| *** shivrao has joined #tripleo | 03:03 | |
| openstackgerrit | Merged openstack/diskimage-builder: Properly fail/trap in eval_run_d https://review.openstack.org/259734 | 03:04 |
| openstackgerrit | Merged openstack/diskimage-builder: Add 1.16.0 releasenotes https://review.openstack.org/316532 | 03:05 |
| *** coolsvap has joined #tripleo | 03:11 | |
| *** coolsvap has quit IRC | 03:15 | |
| *** coolsvap has joined #tripleo | 03:32 | |
| openstackgerrit | Ian Wienand proposed openstack/diskimage-builder: centos-minimal: can be used with base https://review.openstack.org/294385 | 03:37 |
| openstackgerrit | Merged openstack/diskimage-builder: centos-minimal: can be used with base https://review.openstack.org/294385 | 03:46 |
| *** links has joined #tripleo | 03:49 | |
| *** shivrao has quit IRC | 04:03 | |
| *** shivrao has joined #tripleo | 04:09 | |
| *** jobewan has joined #tripleo | 04:10 | |
| *** shivrao has quit IRC | 04:11 | |
| *** anshul has joined #tripleo | 04:12 | |
| *** anshul is now known as Guest28905 | 04:12 | |
| *** jobewan has quit IRC | 04:29 | |
| *** jcoufal has joined #tripleo | 04:46 | |
| openstackgerrit | Ian Wienand proposed openstack/diskimage-builder: Small doc update, add link to relnotes https://review.openstack.org/316576 | 05:15 |
| *** rcernin has joined #tripleo | 05:16 | |
| *** oshvartz has joined #tripleo | 05:20 | |
| *** olap has quit IRC | 05:20 | |
| *** rcernin has quit IRC | 05:21 | |
| *** saneax_AFK is now known as saneax | 05:22 | |
| openstackgerrit | Merged openstack/diskimage-builder: Small doc update, add link to relnotes https://review.openstack.org/316576 | 05:29 |
| *** rcernin has joined #tripleo | 05:38 | |
| *** ramishra has quit IRC | 05:49 | |
| *** Slower has quit IRC | 05:49 | |
| openstackgerrit | Juan Antonio Osorio Robles proposed openstack/puppet-tripleo: Create keystone endpoints in keystone profile https://review.openstack.org/304125 | 05:49 |
| *** ramishra has joined #tripleo | 05:50 | |
| openstackgerrit | Merged openstack/tripleo-image-elements: Don't send duplicate signals via 99-refresh-completed https://review.openstack.org/314226 | 05:51 |
| *** liverpooler has joined #tripleo | 05:55 | |
| openstackgerrit | Merged openstack/os-cloud-config: Replace deprecated LOG.warn with LOG.warning https://review.openstack.org/291568 | 05:55 |
| openstackgerrit | Merged openstack/python-tripleoclient: Updated from global requirements https://review.openstack.org/268528 | 05:56 |
| *** mikelk has joined #tripleo | 05:56 | |
| openstackgerrit | Merged openstack/os-collect-config: Updated from global requirements https://review.openstack.org/264446 | 05:56 |
| *** apetrich_ has joined #tripleo | 06:02 | |
| openstackgerrit | Merged openstack/os-apply-config: Updated from global requirements https://review.openstack.org/303136 | 06:09 |
| openstackgerrit | Merged openstack/tripleo-common: Updated from global requirements https://review.openstack.org/313777 | 06:09 |
| *** jaosorior has joined #tripleo | 06:18 | |
| *** shivrao has joined #tripleo | 06:39 | |
| *** paramite has joined #tripleo | 06:45 | |
| *** shivrao has quit IRC | 06:50 | |
| *** mcornea has joined #tripleo | 06:51 | |
| *** jaosorior has quit IRC | 06:55 | |
| *** jaosorior_ has joined #tripleo | 06:55 | |
| jaosorior_ | Does anybody have an idea why tihs would be happening in the CI? http://logs.openstack.org/27/291627/1/check-tripleo/gate-tripleo-ci-f22-nonha/a53c54b/console.html#_2016-05-16_06_51_35_424 | 06:57 |
| *** jaosorior_ is now known as jaosorior | 07:00 | |
| *** milan has joined #tripleo | 07:03 | |
| openstackgerrit | Babu Shanmugam proposed openstack/puppet-tripleo: Add OVN support for neutron profile https://review.openstack.org/314875 | 07:19 |
| *** ebarrera has joined #tripleo | 07:19 | |
| *** jtomasek has joined #tripleo | 07:26 | |
| *** bvandenh has joined #tripleo | 07:26 | |
| *** olap has joined #tripleo | 07:26 | |
| EmilienM | jaosorior: it might be related to https://review.openstack.org/#/c/312725/ | 07:29 |
| *** zoli_gone-proxy is now known as zoliXXL | 07:32 | |
| zoliXXL | good morning | 07:35 |
| *** stendulker has joined #tripleo | 07:36 | |
| openstackgerrit | Andreas Florath proposed openstack/diskimage-builder: partitioning-parted: extraced and adapted element https://review.openstack.org/313938 | 07:36 |
| *** aufi has joined #tripleo | 07:38 | |
| *** shardy has joined #tripleo | 07:40 | |
| *** itamarl has joined #tripleo | 07:43 | |
| *** dsariel has joined #tripleo | 07:48 | |
| *** ohamada has joined #tripleo | 07:50 | |
| *** hjensas has joined #tripleo | 07:54 | |
| *** hjensas has joined #tripleo | 07:54 | |
| *** hewbrocc` is now known as hewbrocca | 07:59 | |
| *** zoliXXL is now known as zoli|afk | 08:02 | |
| *** jaosorior has quit IRC | 08:03 | |
| *** jaosorior has joined #tripleo | 08:03 | |
| *** dbecker has joined #tripleo | 08:05 | |
| *** mbound has joined #tripleo | 08:09 | |
| *** dsariel has quit IRC | 08:13 | |
| *** mbound has quit IRC | 08:14 | |
| *** jistr has joined #tripleo | 08:30 | |
| *** dmk0202 has joined #tripleo | 08:32 | |
| *** zoli|afk is now known as zoli | 08:37 | |
| openstackgerrit | Merged openstack/instack-undercloud: Create firewall rules with puppet https://review.openstack.org/308374 | 08:39 |
| openstackgerrit | Merged openstack/instack-undercloud: Don't use os-cloud-config/common-venv elements https://review.openstack.org/308550 | 08:40 |
| openstackgerrit | Merged openstack/instack-undercloud: Move persistent journal to puppet https://review.openstack.org/310564 | 08:40 |
| openstackgerrit | Merged openstack/instack-undercloud: Use tripleo::selinux https://review.openstack.org/310565 | 08:42 |
| *** derekh has joined #tripleo | 08:45 | |
| *** panda has quit IRC | 08:48 | |
| *** panda has joined #tripleo | 08:48 | |
| *** xinwu has quit IRC | 08:49 | |
| *** paramite has quit IRC | 08:51 | |
| *** paramite has joined #tripleo | 08:53 | |
| openstackgerrit | Merged openstack/tripleo-heat-templates: Add database directory mount for openvswitchdb https://review.openstack.org/291935 | 08:54 |
| *** electrofelix has joined #tripleo | 08:59 | |
| *** sambetts|afk is now known as sambetts | 09:07 | |
| *** xinwu has joined #tripleo | 09:19 | |
| openstackgerrit | Sam Betts proposed openstack/diskimage-builder: Add dhcp-all-interfaces.target for syncing units https://review.openstack.org/314086 | 09:20 |
| openstackgerrit | Andreas Florath proposed openstack/diskimage-builder: WIP! Add: lvm element https://review.openstack.org/316529 | 09:22 |
| *** dsariel has joined #tripleo | 09:27 | |
| shardy | "could not open disk image /var/lib/libvirt/images/seed_2.qcow2: not in qcow2 format" | 09:34 |
| shardy | uh-oh | 09:34 |
| shardy | CI was looking so green today too :( | 09:34 |
| shardy | derekh: Hey, have we just cut over to using the raw image pabelanger was working on? | 09:36 |
| shardy | suddenly seeing a bunch of failures related to image format | 09:36 |
| derekh | shardy: looking, we shouldn't | 09:37 |
| derekh | shardy: I mean the patch merged yesterday but it shouldn't effecting the jobs running on fedora | 09:38 |
| derekh | slaves | 09:38 |
| shardy | derekh: Yeah, I saw the patch landed but it doesn't seem to correlate with the failures I've looked at | 09:39 |
| shardy | Hmm, actually it's not consistent - looking at the nonha job, two failed due to that since 9am, but the other two failures had a different cause | 09:41 |
| derekh | shardy: can you point me at one that failed with the error | 09:41 |
| shardy | http://logs.openstack.org/11/231211/6/check-tripleo/gate-tripleo-ci-f22-ha/9865e61/console.html | 09:42 |
| hewbrocca | zoli: Hey I was going to ask | 09:42 |
| shardy | all of the HA job failures (3 out of 4) since 9am failed like that | 09:42 |
| shardy | and two of the nonha ones | 09:42 |
| hewbrocca | I know you put in for a replacement motherboard for the TripleO CI bastion host | 09:42 |
| hewbrocca | zoli: any way to tell if any of the other hosts have the same problem | 09:42 |
| shardy | strangle that one HA job passed tho | 09:42 |
| *** bvandenh has quit IRC | 09:44 | |
| zoli | hewbrocca, reading back | 09:44 |
| shardy | upgrades job also has about half of the recent jobs failing in the same way | 09:44 |
| derekh | shardy: don't know whats wrong yet, but I know why your only seeing it on some jobs | 09:46 |
| derekh | shardy: a few lines up from the error you pasted you'll see | 09:46 |
| derekh | 2016-05-16 09:13:24.279 | dd: failed to open 'instack.qcow2': No such file or directory | 09:46 |
| derekh | shardy: the build of the instack.qcow2 image is failing | 09:47 |
| derekh | shardy: we'll only see this on certain repositories, because some use the cache and don't attempt to build it | 09:47 |
| shardy | aha | 09:48 |
| derekh | shardy: so the error will only happen on repositories in the list on line 149 here | 09:48 |
| derekh | http://git.openstack.org/cgit/openstack-infra/tripleo-ci/tree/scripts/common_functions.sh#n149 | 09:48 |
| hewbrocca | Aside from this, how is the rack doing with all the fancy upgrades | 09:48 |
| shardy | hewbrocca: the performance overall looks quite a bit better :) | 09:49 |
| hewbrocca | eeexcellent | 09:49 |
| *** zoli is now known as zoli|lunch | 09:51 | |
| shardy | Some of the nonha jobs have been down around 1hr 10, which is a great improvement, but some are still taking over 1hr 30, so we probably need to investigate and see if there's any scope for tuning further to get more consistent runtimes | 09:51 |
| *** dtantsur is now known as dtantsur|brb | 09:53 | |
| *** tosky has joined #tripleo | 09:58 | |
| derekh | shardy: I'm thinking this patch might have started it https://review.openstack.org/#/c/259734/11 | 09:59 |
| derekh | shardy: maybe not but I'm gonna try a revert now | 09:59 |
| openstackgerrit | Derek Higgins proposed openstack/diskimage-builder: Nothing to see here https://review.openstack.org/139627 | 10:01 |
| shardy | derekh: ack, looks suspicious - failing CI as well :( | 10:01 |
| derekh | shardy: got a meeting now, will check back on ^ in a bit | 10:01 |
| shardy | derekh: ack, thanks for looking into it | 10:01 |
| openstackgerrit | Marios Andreou proposed openstack/puppet-tripleo: Adds the puppet-tripleo manifests for manila https://review.openstack.org/313527 | 10:04 |
| *** xinwu has quit IRC | 10:05 | |
| *** dc has joined #tripleo | 10:14 | |
| *** mgould has joined #tripleo | 10:23 | |
| *** paramite is now known as paramite|afk | 10:33 | |
| *** spredzy has quit IRC | 10:36 | |
| openstackgerrit | Dougal Matthews proposed openstack/tripleo-docs: Deploy and Scale Swift https://review.openstack.org/293311 | 10:44 |
| *** rodrigods has quit IRC | 10:48 | |
| *** rodrigods has joined #tripleo | 10:48 | |
| *** weshay has quit IRC | 10:50 | |
| *** honza has quit IRC | 10:56 | |
| sshnaidm | shardy, derekh , can you please merge it? https://review.openstack.org/#/c/314468/ it's required for log indexing | 10:57 |
| *** zoli|lunch is now known as zoli | 11:00 | |
| *** zoli is now known as zoliXXL | 11:00 | |
| *** spredzy has joined #tripleo | 11:07 | |
| *** stendulker has quit IRC | 11:08 | |
| *** olap has quit IRC | 11:10 | |
| *** olap has joined #tripleo | 11:12 | |
| *** jtomasek_ has joined #tripleo | 11:14 | |
| *** bvandenh has joined #tripleo | 11:18 | |
| derekh | sshnaidm: done | 11:20 |
| *** lucasagomes is now known as lucas-hungry | 11:21 | |
| openstackgerrit | Merged openstack-infra/tripleo-ci: Add timestamps to postci log https://review.openstack.org/314468 | 11:22 |
| jistr | derekh: FYI the revert you posted seems to fix failing instack-virt-setup for me. Didn't try overcloud images build (i'm deploying stable/liberty at this occasion, so i wouldn't hit the problem). | 11:26 |
| sshnaidm | derekh, thanks, FYI - I've opened a bug in the morning: https://bugs.launchpad.net/tripleo/+bug/1582115 as I saw you already looked at it | 11:27 |
| openstack | Launchpad bug 1582115 in tripleo "Periodic jobs fail, because there is no instack.qcow2 image" [Undecided,New] | 11:27 |
| jistr | derekh: thanks for looking into that issue | 11:27 |
| derekh | jistr: sshnaidm ack thanks, I'll let ci on it finish then and will resubmit the revert with a proper commit message to merge it | 11:28 |
| *** dtantsur|brb is now known as dtantsur | 11:28 | |
| *** bvandenh has quit IRC | 11:29 | |
| dtantsur | folks, mind reviewing https://review.openstack.org/#/c/309521/ please? it finally passed the gate and it's a dependency for more nodes-related work | 11:31 |
| *** bvandenh has joined #tripleo | 11:32 | |
| *** jtomasek_ has quit IRC | 11:38 | |
| *** weshay has joined #tripleo | 11:42 | |
| *** MaxPC has joined #tripleo | 11:43 | |
| *** rhallisey has joined #tripleo | 11:44 | |
| *** bvandenh has quit IRC | 11:51 | |
| *** Goneri has quit IRC | 11:53 | |
| *** honza has joined #tripleo | 11:57 | |
| *** dprince has joined #tripleo | 11:59 | |
| *** liverpooler has quit IRC | 12:07 | |
| *** jtomasek_ has joined #tripleo | 12:10 | |
| derekh | jistr: shardy sshnaidm that revert passed CI, going to resubmit the same thing with a proper commit message | 12:11 |
| jistr | cool, thanks | 12:11 |
| *** jayg|g0n` is now known as jayg | 12:12 | |
| openstackgerrit | Derek Higgins proposed openstack/diskimage-builder: Revert "Properly fail/trap in eval_run_d" https://review.openstack.org/316789 | 12:13 |
| derekh | shardy: jistr sshnaidm ^ that should fix it, the exact same patch(with a different commit message passed here https://review.openstack.org/#/c/139627/ ) | 12:14 |
| *** saneax is now known as saneax_AFK | 12:14 | |
| derekh | actually let me add the bug number | 12:16 |
| *** paramite|afk is now known as paramite | 12:16 | |
| openstackgerrit | Derek Higgins proposed openstack/diskimage-builder: Revert "Properly fail/trap in eval_run_d" https://review.openstack.org/316789 | 12:16 |
| *** myoung|lunch has joined #tripleo | 12:21 | |
| *** ooolpbot has joined #tripleo | 12:21 | |
| ooolpbot | URGENT TRIPLEO TASKS NEED ATTENTION | 12:21 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1582115 | 12:21 |
| openstack | Launchpad bug 1582115 in tripleo "Periodic jobs fail, because there is no instack.qcow2 image" [Critical,Triaged] - Assigned to Derek Higgins (derekh) | 12:21 |
| *** ooolpbot has quit IRC | 12:21 | |
| derekh | shardy: I think we're good to merge ^^ without waiting for CI if you want, its already passed in the test patch | 12:22 |
| * derekh goes to lunch | 12:22 | |
| *** coolsvap has quit IRC | 12:22 | |
| *** ohamada has quit IRC | 12:22 | |
| *** ohamada has joined #tripleo | 12:22 | |
| *** coolsvap has joined #tripleo | 12:22 | |
| *** lucas-hungry is now known as lucasagomes | 12:26 | |
| *** bvandenh has joined #tripleo | 12:27 | |
| *** ohamada_ has joined #tripleo | 12:29 | |
| *** ohamada has quit IRC | 12:29 | |
| *** ohamada__ has joined #tripleo | 12:29 | |
| *** ohamada_ has quit IRC | 12:29 | |
| *** trown|brb is now known as trown | 12:31 | |
| *** zaneb has joined #tripleo | 12:32 | |
| sshnaidm | trown, hi | 12:33 |
| trown | sshnaidm: morning | 12:33 |
| *** coolsvap has quit IRC | 12:33 | |
| sshnaidm | trown, actually the noon :) | 12:33 |
| trown | sshnaidm: or UGT morning :) http://www.geekality.net/2010/05/14/universal-greeting-time-ugt/ | 12:34 |
| sshnaidm | trown, great idea :) | 12:34 |
| sshnaidm | trown, I try to run quickstart and overcloud deploy fails, do you know about any blockers for now? it's here: https://ci.centos.org/job/sshnaidm_quickstart_tripleoci/2/consoleFull | 12:35 |
| sshnaidm | trown, the config is simple: https://paste.fedoraproject.org/367085/34022351/ | 12:37 |
| trown | sshnaidm: looks like that job is using the testing image: http://artifacts.ci.centos.org/artifacts/rdo/images/master/delorean/testing/undercloud.qcow2 | 12:37 |
| sshnaidm | trown, right | 12:38 |
| trown | sshnaidm: which does not look to have passed CI for HA for a few days | 12:38 |
| trown | sshnaidm: I was on PTO all last week, so not sure what is up with that | 12:38 |
| openstackgerrit | Marios Andreou proposed openstack/puppet-tripleo: Adds the puppet-tripleo manifests for manila https://review.openstack.org/313527 | 12:38 |
| sshnaidm | trown, could you please explain me briefly the point with testing/stable/passed-ci/current.. images? I think I don't understand it well. Either IRC or Blujeans, as you wish - in your time | 12:39 |
| *** julim has joined #tripleo | 12:40 | |
| *** Guest28905 has quit IRC | 12:40 | |
| trown | sshnaidm: testing image corresponds to current (actually consistent but they should be the same most of the time) and is used in the promote pipeline | 12:41 |
| trown | sshnaidm: if promote passes, that image is copied to stable, and the repo is symlinked to current-passed-ci | 12:42 |
| sshnaidm | trown, what is promote pipeline again? | 12:43 |
| trown | sshnaidm: https://ci.centos.org/view/rdo/view/promotion-pipeline/ | 12:43 |
| trown | sshnaidm: it is what we use in RDO to promote the trunk repos | 12:43 |
| *** rlandy has joined #tripleo | 12:44 | |
| sshnaidm | trown, do you have any links maybe that describe the process? I try to catch the whole picture.. | 12:45 |
| trown | sshnaidm: on that page we can see that the HA job on master has not passed for 4.5 days, so I would not expect the testing image to work... | 12:45 |
| sshnaidm | trown, and mitaka is 7 days? | 12:45 |
| trown | sshnaidm: not sure what you are looking for... if you click on the multijob itself it shows a better view of the flow https://ci.centos.org/view/rdo/view/promotion-pipeline/job/rdo-delorean-promote-master/ | 12:45 |
| *** rbrady has joined #tripleo | 12:46 | |
| trown | sshnaidm: ya, but mitaka is not failing because of tripleo... need to ask dmsimard to look at why that one mitaka job is failing | 12:46 |
| *** morazi has joined #tripleo | 12:46 | |
| *** rbrady has quit IRC | 12:48 | |
| *** ohamada__ has quit IRC | 12:48 | |
| *** ohamada__ has joined #tripleo | 12:48 | |
| *** rbrady has joined #tripleo | 12:48 | |
| sshnaidm | trown, is there easy way to see all patches between last success and failure in promote-ha job? | 12:48 |
| sshnaidm | trown, afaiu it's between two give delorean hashes | 12:49 |
| sshnaidm | s/give/given/ | 12:49 |
| dmsimard | trown: oh hai welcome back | 12:50 |
| *** zoliXXL is now known as zoli|afk | 12:50 | |
| *** jdob has joined #tripleo | 12:50 | |
| trown | sshnaidm: ya, when it hasnt passed for a long time, it is pretty futile to take that approach, but https://trunk.rdoproject.org/centos7-master/report.html shows each build | 12:50 |
| trown | dmsimard: thanks! | 12:50 |
| EmilienM | just an FYI for folks running tripleo using nova from trunk: I found https://bugs.launchpad.net/nova/+bug/1582103 | 12:52 |
| openstack | Launchpad bug 1582103 in OpenStack Compute (nova) "nova-novncproxy fails to start: NoSuchOptError: no such option in group DEFAULT: verbose" [Undecided,In progress] - Assigned to Emilien Macchi (emilienm) | 12:52 |
| *** Guest28905 has joined #tripleo | 12:52 | |
| sshnaidm | trown, dmsimard I look at http://trunk.rdoproject.org/centos7-master/report.html , but it's limited. Is there any continuation or other option to match between delorean hash and patch? | 12:55 |
| *** links has quit IRC | 12:55 | |
| *** tzumainn has joined #tripleo | 12:56 | |
| *** MaxPC has quit IRC | 12:57 | |
| *** MaxPC has joined #tripleo | 12:57 | |
| trown | sshnaidm: well delorean hash is the git hash of the project that created it | 12:57 |
| *** myoung|lunch has quit IRC | 12:58 | |
| sshnaidm | trown, this i.e.: f103c8797a55aec30a1857470c7cb4728a016a70_0fd1a33c first part is git, and after _ ? | 12:58 |
| trown | sshnaidm: and there is a versions.csv in every repo that shows the git hashes for all packages in that repo https://trunk.rdoproject.org/centos7-master/d5/d2/d5d2de7719c0b908617122c6b24467ce15e35de0_13b49d40/versions.csv | 12:58 |
| trown | sshnaidm: ya it is <sha1 of code repo>_<short sha1 of packaging repo> | 12:59 |
| sshnaidm | trown, cool, thanks | 12:59 |
| *** athomas has quit IRC | 13:01 | |
| *** Goneri has joined #tripleo | 13:03 | |
| *** Guest28905 has quit IRC | 13:09 | |
| *** ooolpbot has joined #tripleo | 13:10 | |
| ooolpbot | URGENT TRIPLEO TASKS NEED ATTENTION | 13:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1582115 | 13:10 |
| *** ooolpbot has quit IRC | 13:10 | |
| openstack | Launchpad bug 1582115 in tripleo "Periodic jobs fail, because there is no instack.qcow2 image" [Critical,Triaged] - Assigned to Derek Higgins (derekh) | 13:10 |
| EmilienM | jaosorior, trown: could we merge 2 patches that need to land together? | 13:10 |
| EmilienM | https://review.openstack.org/#/c/316361/ and https://review.openstack.org/#/c/316360/ | 13:10 |
| EmilienM | oh, marios -1 one of them | 13:11 |
| EmilienM | nevermind, I need to send a new PS.. | 13:12 |
| derekh | All, this appears to fix that ^^^ https://review.openstack.org/#/c/316789/ | 13:12 |
| trown | derekh: you had me at "it merged without passing CI" :) | 13:13 |
| derekh | ;-) | 13:14 |
| EmilienM | marios: please review https://review.openstack.org/#/c/316360/ again, your comments are for keystone, not for nova | 13:14 |
| *** lblanchard has joined #tripleo | 13:14 | |
| marios | EmilienM: ack will do | 13:14 |
| EmilienM | derekh: gate-tripleo-ci-f22-nonha was failing on https://review.openstack.org/#/c/259734/ | 13:14 |
| EmilienM | marios: and please +2 if you can, so we can land it now | 13:15 |
| trown | derekh: that is ok to merge the revert without CI? | 13:15 |
| openstackgerrit | Ryan Brady proposed openstack/tripleo-common: Initial Integration of Mistral into TripleO https://review.openstack.org/282366 | 13:15 |
| EmilienM | trown: I think so, the job were failing too | 13:15 |
| derekh | trown: EmilienM yes, when I wasn't sure I tested it here https://review.openstack.org/#/c/139627/ | 13:16 |
| EmilienM | derekh: can we land your patch? | 13:16 |
| EmilienM | oh but gate-dib-dsvm-functests-centos-7 is failing | 13:16 |
| derekh | EmilienM: trown looking at the failure to make sure | 13:17 |
| trown | ok I will let someone else pull the +A trigger on that :) | 13:17 |
| EmilienM | trown: we can't merge it, it's not passing gate | 13:18 |
| trown | DIB has gates? I thought it was like the rest of tripleo with only check queue | 13:18 |
| EmilienM | http://logs.openstack.org/89/316789/2/check/gate-dib-dsvm-functests-centos-7/9cda35f/console.html#_2016-05-16_12_39_54_476 | 13:19 |
| EmilienM | maybe we can recheck | 13:19 |
| derekh | trown: EmilienM the tripleo job failed because it was longer then 20 minutes waiting for a testenv, so not related | 13:20 |
| derekh | 2016-05-16 13:05:25.698 | + sudo kill -9 16096 | 13:20 |
| marios | EmilienM: can you check the enable-tsl env file (commented on review) | 13:20 |
| EmilienM | marios: oh right | 13:21 |
| derekh | not sure about the gate job but I think we're ok to ignore the tripleo job failure and not wait on it, based on it passing in the test patch and others has seen it fix the problem localy | 13:21 |
| *** Guest28905 has joined #tripleo | 13:22 | |
| openstackgerrit | Emilien Macchi proposed openstack/tripleo-heat-templates: Remove Nova EC2 deployment https://review.openstack.org/316360 | 13:24 |
| EmilienM | marios: thx ^ should be ok now, let's wait CI and land it today if possible | 13:24 |
| marios | EmilienM: ack thx | 13:24 |
| openstackgerrit | Paul Belanger proposed openstack/diskimage-builder: Fix path issue for locale-archive.tmpl https://review.openstack.org/316831 | 13:24 |
| marios | EmilienM: i see you also ran the build_endpoint_map.py this time :) | 13:27 |
| EmilienM | marios: is it ok? | 13:27 |
| EmilienM | so we clean all nova ec2 stuffs in one shot | 13:28 |
| marios | EmilienM: I *tuink* so | 13:28 |
| marios | yeah | 13:28 |
| EmilienM | and I'll do keystine next time | 13:28 |
| EmilienM | btw, puppet openstack is preparing a module for ec2api | 13:28 |
| EmilienM | it's almost ready, I think it will be working during Newton | 13:28 |
| *** egafford has joined #tripleo | 13:30 | |
| *** jprovazn has quit IRC | 13:30 | |
| EmilienM | marios, jistr: Also, I did some cleanup in instack, we can merge 2 patches but they need to land together | 13:30 |
| EmilienM | otherwise we will break CI again | 13:30 |
| EmilienM | https://review.openstack.org/#/c/315238/ first (even if failing) and https://review.openstack.org/#/c/316112/ that test the first patch | 13:31 |
| EmilienM | it won't break overcloud since we don't install ironic on it | 13:31 |
| EmilienM | marios, jistr: or I can put the Depends-On the other way around | 13:37 |
| EmilienM | so we reduce the downtime | 13:37 |
| openstackgerrit | Emilien Macchi proposed openstack/instack-undercloud: Cleanup Puppet code that deploys Ironic https://review.openstack.org/315238 | 13:38 |
| *** egafford1 has joined #tripleo | 13:38 | |
| jistr | EmilienM, marios: hmm i don't think there's a necessary dependency between the two though, no? we could land the big refactoring and if we just keep this line: | 13:40 |
| jistr | ironic_config { | 13:40 |
| jistr | 'DEFAULT/my_ip': value => hiera('controller_host'); | 13:40 |
| jistr | it should pass all CI jobs on tripleo, no? | 13:40 |
| jistr | and then we can have 2 mini patches (puppet + tht) for the my_ip specifically, which would have this chicken-and-egg problem | 13:41 |
| EmilienM | jistr: nope, puppet-ironic patch contains the my_ip patch | 13:41 |
| EmilienM | I'm afk for lunch, I just want to move forward with the patches, thanks. | 13:41 |
| jistr | yea but that's the only thing that the puppet-ironic patch contains... | 13:41 |
| *** egafford has quit IRC | 13:42 | |
| jistr | my point is why would we do such a big refactoring with a red CI, as i see we could make it green easily by making the my_ip switch a separate patch | 13:42 |
| jrist | morning/afternoon all | 13:43 |
| hewbrocca | where is jstir | 13:47 |
| hewbrocca | we should go for the trifecta | 13:47 |
| jistr | :)) | 13:47 |
| jistr | EmilienM: flipping the depends-on WFM too :) Thanks, +2 | 13:48 |
| dtantsur | shardy, hi! mind taking one more look at https://review.openstack.org/#/c/309521/ please? | 13:50 |
| pabelanger | Morning! | 13:51 |
| pabelanger | I'd like to migrate tripleo to centos-7 jenkins slaves today. Could people please review and +1 https://review.openstack.org/#/c/316856/ | 13:52 |
| pabelanger | derekh: are okay with bumping the HDD space on h1 from 20GB to 40GB? | 13:53 |
| *** olap has quit IRC | 13:53 | |
| derekh | pabelanger: we wont be able hold enough of that size if we want to switch to the SSD's | 13:55 |
| *** olap has joined #tripleo | 13:55 | |
| pabelanger | derekh: which size do you think you'll be able to support? | 13:55 |
| shardy | dtantsur: done - just one comment - is there scope to refactor some of these things into ironicclient long-term? | 13:55 |
| shardy | it seems like several of those functions are pretty generic and not really tied to tripleo | 13:56 |
| shardy | e.g nodes.wait_for_provision_state seems like a pretty common pattern for all users of ironic? | 13:56 |
| derekh | pabelanger: 25G I think, but we should look into using the SSD's as a cache for the bigger spinning disks, then we can bump past 40G if you want | 13:56 |
| pabelanger | derekh: okay, 25GB is a start. Lets see how that works | 13:57 |
| derekh | pabelanger: ok | 13:57 |
| dtantsur | shardy, some of them, yes (I hope) | 13:57 |
| dtantsur | shardy, it's on my radar, just overloaded with other things | 13:57 |
| shardy | dtantsur: ack, sounds good - perhaps we could start an etherpad and/or raise some wishlist bugs with details of what we hope to refactor out of tripleo-common in due course | 13:58 |
| dtantsur | also we should merge https://review.openstack.org/315238 ASAP, cause ironic is removing the deprecated parameters that we're using | 13:58 |
| dtantsur | shardy++ | 13:58 |
| *** trozet has joined #tripleo | 13:59 | |
| shardy | dtantsur: aslo IIRC there was talk of aligning the nodes json file more closely with ironic recently - that seems like an ideal opportunity to switch formats and move handling node registration into ironicclient to me? | 13:59 |
| dtantsur | shardy, yes (that's actually a prerequisite for that) | 13:59 |
| dtantsur | again, someone just has to do it :) maybe you can bribe mgould ;) | 14:00 |
| EmilienM | jistr: thanks | 14:00 |
| shardy | dtantsur: heh - perhaps we can raise some wishlist bugs and tag them ironic-cleanup or something, then find folks willing to help | 14:00 |
| dtantsur | yep | 14:00 |
| *** _prad_ has joined #tripleo | 14:02 | |
| * mgould hears the sound of him being volunteered for something | 14:03 | |
| *** coolsvap has joined #tripleo | 14:03 | |
| mgould | what am I being volunteered for, and how much will I hate it? | 14:03 |
| dtantsur | VERY MUCH | 14:04 |
| * mgould runs away | 14:04 | |
| mgould | unifying the JSON formats that ironic and tripleo use? | 14:04 |
| derekh | greghaynes: FYI we're about to revert this DIB patch https://review.openstack.org/#/c/316789/ | 14:05 |
| *** sshnaidm has quit IRC | 14:05 | |
| dprince | shardy, d0ugal: hi re the comments here I'm unclear why we even need this patch https://review.openstack.org/#/c/309521/6 vs. just doing what I did in baremetal.yaml | 14:06 |
| dprince | shardy, d0ugal: https://review.openstack.org/#/c/300200/2/workflows/baremetal.yaml | 14:06 |
| pabelanger | derekh: what is the error you are seeing? | 14:06 |
| ayoung | trown, I have an appraoch to making quickstart work for multiple undercloud nodes instead of just one. https://review.openstack.org/#/c/315749/ One thing I need to sort is how to trigger adding the "identity" server, which can be added to the list of undercloud nodes | 14:06 |
| dprince | shardy, d0ugal; copying around python code is the pattern I'd like to avoid... without seeing what/where we are actually going to use the code | 14:06 |
| ayoung | Ideally it would be a -e option to trigger it. Any suggestions? | 14:07 |
| *** sshnaidm has joined #tripleo | 14:07 | |
| derekh | pabelanger: just this at the end of the image build "/tmp/image.e3mjIkbX/mnt is not a directory" | 14:07 |
| pabelanger | derekh: is the log online? Curious if it is the same issue we are seeing on nodepool.o.o | 14:08 |
| d0ugal | dprince: hmm, good point. I have not looked through /c/300200 fully yet. | 14:08 |
| dprince | shardy, d0ugal; what I'm getting at... is that if we can demonstrate a workflow that accomplishes baremetal introspection, and wire it into python-tripleoclient I'd like to stop this sort of pattern I think | 14:08 |
| derekh | pabelanger: one sec | 14:08 |
| rbrady | flfuchs, jtomasek: I'm planning to bring up the mistral bug at the IRC meeting today. If you have any other additional notes than what is in https://bugs.launchpad.net/mistral/+bug/1581649 , please update the bug prior to 1600 UTC | 14:09 |
| openstack | Launchpad bug 1581649 in Mistral "Action Execution Response Timeout" [Undecided,New] | 14:09 |
| derekh | pabelanger: bottom of http://logs.openstack.org/11/231211/6/check-tripleo/gate-tripleo-ci-f22-ha/9865e61/logs/instack-build.txt.gz | 14:09 |
| d0ugal | dprince: Sure, I think part of the goal is to move the code to tripleo-common so it can be used via mistral and then CLI can call that instead | 14:09 |
| dprince | d0ugal: I spoke w/ rbrady and I think I'm going to take another pass on baremetal.yaml this week. I will likely fix or reply to all your comments, and probably add in zaqare queues so that we don't loose the ability to have fast asycn communication with the end user | 14:09 |
| *** zoli|afk is now known as zoli | 14:09 | |
| dprince | d0ugal: I view the goal as having "API driven" workflows. And if we don't need actual code to do that then whatever | 14:10 |
| d0ugal | dprince: but I have to say I am a bit confused about tripleo-common as it seems it is being used differently | 14:10 |
| d0ugal | dprince: Yup, makes sense | 14:10 |
| *** ooolpbot has joined #tripleo | 14:10 | |
| ooolpbot | URGENT TRIPLEO TASKS NEED ATTENTION | 14:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1582115 | 14:10 |
| *** ooolpbot has quit IRC | 14:10 | |
| openstack | Launchpad bug 1582115 in tripleo "Periodic jobs fail, because there is no instack.qcow2 image" [Critical,Triaged] - Assigned to Derek Higgins (derekh) | 14:10 |
| shardy | dtantsur: https://bugs.launchpad.net/tripleo/+bug/1582250 | 14:10 |
| openstack | Launchpad bug 1582250 in tripleo "Refactor common things from tripleo-common to ironicclient" [Wishlist,Triaged] | 14:10 |
| dtantsur | thnx | 14:10 |
| shardy | https://bugs.launchpad.net/tripleo/+bugs?field.tag=ironic-cleanups | 14:10 |
| shardy | feel free to raise any others and tag them with ironic-cleanups | 14:10 |
| openstackgerrit | Merged openstack/tripleo-common: Import initial_state logic from tripleoclient https://review.openstack.org/309521 | 14:11 |
| dprince | d0ugal: it may be we need an extra custom action or two. And that is fine, but we need to stop the python code copying I think. Because tripleo-common is no longer meant to be just a python library. | 14:11 |
| d0ugal | dprince: Right, then we need to tell everyone to stop writing code for it like a library, I'd guess 50% of the open patches are doing that. | 14:12 |
| pabelanger | derekh: Ya, not that same. We're on centos-minimal now | 14:12 |
| dprince | d0ugal: yeah, like I thought we were clear on this. Like I feel like we are basically a lot of python code into tripleo-common that will basically be unused when we actually start using things. So lets stop it | 14:13 |
| d0ugal | dprince: k, I think we should bring this up in the meeting tomorrow and/or on the list just to make sure it is clear | 14:14 |
| *** links has joined #tripleo | 14:15 | |
| dprince | d0ugal: anyways, that patch just landed. So I guess I'll take a stab at upgrading the baremetal.yaml workflow and cutting python-tripleoclient to call and use that workflow (similar to this https://review.openstack.org/#/c/313957/ or this https://github.com/dprince/python-tripleoclient/tree/mistral) | 14:16 |
| d0ugal | dprince: Sounds good | 14:16 |
| dprince | d0ugal: is there an etherpad of other code we are copying over that isn't on this: https://etherpad.openstack.org/p/tripleo-mistral-api | 14:16 |
| d0ugal | dprince: No, I didn't even know about that etherpad | 14:17 |
| derekh | pabelanger: ok | 14:17 |
| *** dmacpher has joined #tripleo | 14:17 | |
| pabelanger | okay, I've updated https://review.openstack.org/#/c/316856/ which migrated tripleo to centos-7, and changes tripleo-centos-7 back to h1 (with derekh bumping h1 to 25GB), and sets f22 to min-ready 1 and centos-7 min-ready: 18. We'll then keep f22 around for a few days, and if things are great, removed it by weeks end? | 14:19 |
| EmilienM | marios: thanks for your review, I did a mistake, and you found it | 14:19 |
| openstackgerrit | Emilien Macchi proposed openstack/puppet-tripleo: Composable role for RabbitMQ https://review.openstack.org/311849 | 14:21 |
| openstackgerrit | Emilien Macchi proposed openstack/tripleo-heat-templates: Deploy RabbitMQ as a composable role https://review.openstack.org/311863 | 14:21 |
| openstackgerrit | Jason E. Rist proposed openstack/tripleo-ui: Refactor notification for better display https://review.openstack.org/316874 | 14:22 |
| jrist | jtomasek: ^ | 14:22 |
| jtomasek | jrist: thx | 14:22 |
| dprince | shardy, d0ugal: until we discuss the approach I've block it here: https://review.openstack.org/#/c/309523/3 | 14:22 |
| jrist | jtomasek: warning: it doesn't work quite right... ;) | 14:22 |
| d0ugal | dprince: okay, sounds good. | 14:23 |
| marios | EmilienM: ack np i noticed cos you mentioned the erlang cookie (and i saw the comment on the tht side where it was removed) | 14:23 |
| d0ugal | mgould: ^ see above if you are wondering why you just got a -2 :) | 14:23 |
| *** jstir has joined #tripleo | 14:24 | |
| mgould | d0ugal, thanks for the heads-up | 14:24 |
| *** electrofelix has quit IRC | 14:24 | |
| shardy | dprince: this is related to the discussion I just had with dtantsur - the issue is https://review.openstack.org/#/c/309521/6/tripleo_common/utils/nodes.py doesn't exist in ironicclient | 14:24 |
| shardy | but if that can be replicated directly via mistral, I guess we can move over to that instead | 14:25 |
| dprince | shardy: yeah, like this is the first example I did back in November I think | 14:25 |
| dprince | shardy: rbrady pushed my patch here https://review.openstack.org/#/c/300200 | 14:26 |
| shardy | Ok, that's kinda a step beyond what I orignally asked mgould to do, which was just not to add a bunch of stuff to tripleclient, and instead put it in tripleo-common | 14:26 |
| *** jobewan has joined #tripleo | 14:26 | |
| dprince | shardy: and it needs to be updated but it demonstrates what I think would work. I will try and spend some time on that patch this week perhaps | 14:26 |
| shardy | IIRC at that point it was too early to consider a pure mistral solution, I guess we can now | 14:26 |
| dprince | shardy: pure mistral won't work for all workflows | 14:27 |
| dprince | shardy: it did work quite well (at the time) for this one | 14:27 |
| dprince | shardy: that said, if we need some custom actions that is fine. But lets be selective about that | 14:27 |
| mgould | dprince, shardy: the sequence of events was "your code looks OK, but should go in tripleo-common" "OK, I'll move it" | 14:27 |
| dprince | shardy: and let the workflows themselves drive it | 14:27 |
| mgould | *time passes, oceans rise and fall* | 14:27 |
| EmilienM | marios: ++ thanks a lot... | 14:28 |
| *** jobewan is now known as jobewan_away | 14:28 | |
| shardy | mgould: Yeah, you've done exactly the right thing, it's just that our implementation has evolved since then, apologies | 14:28 |
| jrist | jtomasek: I am/was in the process of migrating some of the control of the notification itself into the notification itself | 14:28 |
| dprince | mgould: yep I got it. That is one approach to doing this. But my concern is that we'll end up with a pile of stuff that never gets used | 14:28 |
| jrist | jtomasek: rather than in the notificationtoaster | 14:28 |
| shardy | personally I'd be fine with just moving the code, then refactoring into mistral, but dprince has a point that we could potentially save some time if we just do it now | 14:28 |
| *** tserong has quit IRC | 14:29 | |
| mgould | shardy, dprince: OK, thanks | 14:29 |
| *** jobewan_away is now known as jobewan | 14:29 | |
| dprince | shardy: moving code was the old TripleO API plan I think. | 14:29 |
| mgould | if https://review.openstack.org/#/c/309523/ isn't going to merge, then we should back out https://review.openstack.org/#/c/309521/ pending a rewrite in Mitral | 14:29 |
| dprince | shardy: we may well need to port some of the code into custom actions still, but I'd like to stress that our work be driven by workflows themselves and not an effort to get rid of python out of python-tripleoclient | 14:30 |
| dprince | shardy: similiar to the demoposable services work in t-h-t, we'll get less code in python-tripleoclient in time I think | 14:30 |
| jrist | jtomasek: so I was going to have the notification time out by itself unless it's hovered and unless there is a dismissable (close button) which is triggered | 14:30 |
| dprince | shardy: by simply focussing on creating the workflows we need, rather than moving code | 14:31 |
| jrist | jtomasek: I upped the timeout to 89000 ms so that I could hover over it and test the results | 14:31 |
| openstackgerrit | Carlos Camacho proposed openstack/tripleo-heat-templates: Composable roles within services - MongoDB https://review.openstack.org/315051 | 14:31 |
| jtomasek | jrist: you're on a good track. you just duplicate some of the properties in notification component state, when it is already available in notification object from app state | 14:31 |
| shardy | dprince: I guess I had a different view on it, which was that moving code was still valid and it would help identify the common chunks of code that need refactoring into either mistral workflows or custom actions, because it'll all be in one place (tripleo-common) vs scattered around -common and the client | 14:32 |
| *** aufi has quit IRC | 14:32 | |
| jistr | derekh, trown: can i get to some prebuilt TripleO overcloud stable images? e.g. something like the images cached from periodic jobs, or if RDO has something to download. Did some searching but no luck so far. | 14:32 |
| jtomasek | jrist: I was thinking about splitting it to 2 parts - 1. get the new styles + notifications listing. 2. handle the disapearing | 14:32 |
| dprince | shardy: it isn't invalid. And it can be done that way but I sort of view it as the long trail, when we could just cut over and use API workflows directly (faster) I think | 14:33 |
| *** veteran has joined #tripleo | 14:34 | |
| *** veteran has quit IRC | 14:34 | |
| *** electrofelix has joined #tripleo | 14:35 | |
| trown | jistr: there are images published for RDO | 14:36 |
| derekh | jistr: the ones tripleo ci uses arn't publicly down loadable at the moment, if its a once off I can get one for you, if its a place you want to get them in general then we would have to decide if thats something we want to do | 14:36 |
| trown | jistr: https://ci.centos.org/artifacts/rdo/images/master/delorean/stable/ is RDO images | 14:36 |
| pabelanger | derekh: rebased | 14:37 |
| trown | jistr: s/master/mitaka/ or s/master/liberty/ as needed | 14:37 |
| jistr | trown: cool, thanks | 14:37 |
| *** rajinir has joined #tripleo | 14:37 | |
| jistr | derekh: i hope what trown posted will do for now, thanks | 14:38 |
| derekh | jistr: | 14:39 |
| derekh | jistr: ok | 14:39 |
| trown | jistr: master hasnt passed CI for a few days, so the stable images are a few days old | 14:39 |
| trown | just fyi | 14:39 |
| derekh | pabelanger: reviewed, thanks | 14:39 |
| shardy | trown: Hey, I've been chatting with pradk re http://lists.openstack.org/pipermail/openstack-dev/2016-May/095097.html | 14:39 |
| pabelanger | derekh: let me know when you bump to 25GB, going to move forward in openstack-infra with 316887 then recheck experimental | 14:40 |
| shardy | trown: I wanted your input from an RDO perspective, as it involves a not-exactly-appropriate-for-stable stable backport (yes, another one!) | 14:40 |
| *** veteran has joined #tripleo | 14:40 | |
| *** veteran has quit IRC | 14:40 | |
| shardy | basically it bakes the switchover to aodh into a stable update, vs handling it as part of the upgrade process to mitaka | 14:40 |
| shardy | in theory it should be backwards compatible, but we've not got sufficient coverage in CI to prove it | 14:41 |
| trown | shardy: I think backports to liberty were not restricted right? | 14:42 |
| *** dprince has quit IRC | 14:42 | |
| *** egafford1 has quit IRC | 14:42 | |
| trown | shardy: and we have it in stable/mitaka so seems fine | 14:42 |
| shardy | trown: well, I originally proposed we'd switch it to match stable/mitaka, e.g normal stable maint rules (mostly because we can't backport anything to liberty that's not first backported to mitaka) | 14:42 |
| shardy | this breaks that, and it's also inconsistent with the current stable-maint move to "critical bugfix only" phase for liberty generally | 14:43 |
| trown | shardy: right I agree we cant backport to liberty if it is not in mitaka, so that protects us from backport everything to liberty | 14:43 |
| derekh | pabelanger: am I googling correctly....no way to update a nova flavor | 14:44 |
| shardy | but it may be justified if it eases upgrades, and you think it looks sane from an RDO perspective, e.g everything's packaged and a minor update switching services around isn't going to break folks | 14:44 |
| shardy | trown: right, this one is in mitaka, so strictly speaking is OK, but it's somewhat risky from an operator impact perspective | 14:44 |
| trown | shardy: ya, aodh is in the liberty repos | 14:44 |
| pabelanger | derekh: okay, guess a new flavor? | 14:45 |
| trown | shardy: reading that mail... it does not look like there is a great way to upgrade without that backport | 14:45 |
| derekh | pabelanger: ya how does h2.ci sound | 14:45 |
| shardy | trown: yeah, although there probably could be a variation on option (2) that uses puppet | 14:46 |
| trown | really need upgrades coverage in RDO promote pipeline | 14:46 |
| *** jprovazn has joined #tripleo | 14:47 | |
| derekh | pabelanger: h2.ci created | 14:47 |
| shardy | trown: do you have a release note medium for RDO TripleO releases, e.g is there some way we can communicate this change? | 14:47 |
| trown | shardy: not really... | 14:47 |
| *** hjensas has quit IRC | 14:47 | |
| *** noslzzp has joined #tripleo | 14:48 | |
| trown | shardy: pradk I am not sure I understand why alarms being down during upgrade is a big problem? | 14:49 |
| pabelanger | derekh: okay, let me update patch | 14:50 |
| shardy | trown: because they'll end up shut down until you've upgraded all compute nodes and run the final puppet converge | 14:50 |
| shardy | trown: which might take days | 14:50 |
| *** ibravo has joined #tripleo | 14:51 | |
| shardy | trown: it does expose a big limitation in our current upgrade architecture tho - it should be possible to specify e.g taking down the old alarm services during or just prior to the final converge step, instead of when the upgrade starts | 14:52 |
| trown | shardy: ah.. that is what I was missing. days is a long time | 14:52 |
| shardy | that may not be possible tho due to package level obsoletes I guess | 14:52 |
| pabelanger | derekh: okay, updated | 14:54 |
| derekh | pabelanger: ack thanks | 14:55 |
| *** jaosorior has quit IRC | 14:55 | |
| *** jaosorior has joined #tripleo | 14:56 | |
| *** liverpooler has joined #tripleo | 14:57 | |
| trown | shardy: pradk does switching to aodh from ceilo alarms have any impact on the user? ie if it happens unintentionally what is worst case? | 14:58 |
| hewbrocca | jstir: thppppppppt | 15:01 |
| jstir | hewbrocca: DIAF | 15:01 |
| hewbrocca | heh | 15:01 |
| *** julim has quit IRC | 15:01 | |
| *** jaosorior has quit IRC | 15:01 | |
| *** itamarl has quit IRC | 15:01 | |
| *** jaosorior has joined #tripleo | 15:02 | |
| *** julim has joined #tripleo | 15:03 | |
| openstackgerrit | Leif Madsen proposed openstack/tripleo-quickstart: Update contributing.md to match reality https://review.openstack.org/314127 | 15:04 |
| *** veteran has joined #tripleo | 15:05 | |
| *** veteran has quit IRC | 15:05 | |
| *** ooolpbot has joined #tripleo | 15:10 | |
| ooolpbot | URGENT TRIPLEO TASKS NEED ATTENTION | 15:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1582115 | 15:10 |
| *** ooolpbot has quit IRC | 15:10 | |
| openstack | Launchpad bug 1582115 in tripleo "Periodic jobs fail, because there is no instack.qcow2 image" [Critical,Triaged] - Assigned to Derek Higgins (derekh) | 15:10 |
| *** _prad_ is now known as pradk- | 15:11 | |
| *** ibravo has quit IRC | 15:14 | |
| *** diazjf has joined #tripleo | 15:19 | |
| *** myoung|lunch has joined #tripleo | 15:23 | |
| *** yamahata has joined #tripleo | 15:25 | |
| *** Slower has joined #tripleo | 15:26 | |
| *** ebarrera has quit IRC | 15:28 | |
| openstackgerrit | Sagi Shnaidman proposed openstack-infra/tripleo-ci: WIP: DONT MERGE Running tests https://review.openstack.org/316436 | 15:32 |
| d0ugal | shardy: Given this was all green on PS29 (and the code hasn't changed other than a comment in the requirements.txt) do you think we can merge? https://review.openstack.org/#/c/282366/ | 15:39 |
| d0ugal | or should I recheck? | 15:39 |
| EmilienM | jistr: hey | 15:40 |
| EmilienM | can we land https://review.openstack.org/#/c/316360/ and right away https://review.openstack.org/#/c/316361/ ? | 15:41 |
| EmilienM | or marios, jaosorior ^ | 15:41 |
| pradk- | trown, shardy, there shouldnt be any impact to the user.. for existing users using ceilo alarms.. ceilometer api and client will auto redirect any alarm calls to aodh | 15:42 |
| pradk- | trown, shardy, so the transition should be barely noticeable | 15:43 |
| *** olap has quit IRC | 15:43 | |
| *** dmk0202 has quit IRC | 15:44 | |
| trown | pradk-: shardy I think backport is probably best option then | 15:44 |
| *** olap has joined #tripleo | 15:46 | |
| jistr | EmilienM: would be good to see HA job pass on https://review.openstack.org/#/c/316360/ first. I saw it passed on 316361, but that was yesterday, so it didn't use the latest patch set of 316360 | 15:47 |
| EmilienM | jistr: it pass with the depends-on, very recently https://review.openstack.org/#/c/316361/ | 15:47 |
| EmilienM | it's safe to merge both IMHO | 15:47 |
| EmilienM | we don't need to run a CI job for it, I think | 15:47 |
| EmilienM | jistr: EC2 is not used anywhere | 15:50 |
| EmilienM | it even does not exist anywhere anymore | 15:50 |
| *** fultonj has joined #tripleo | 15:51 | |
| EmilienM | jistr: can you approve please? | 15:53 |
| jistr | EmilienM: i rechecked, though i'm not holding anyone's hand back in case they want to +A immediately. As i wrote above, i saw that it passed with depends-on, but it wasn't testing the latest patchset. The diff between PS1 and PS2 is 120 lines b/c of endpoint map regeneration only in PS2. https://review.openstack.org/#/c/316360/1..2 | 15:56 |
| EmilienM | jaosorior: your +A is useless, it the other patch is not merged | 15:57 |
| EmilienM | jaosorior: since they don't co-gate, please drop +A so we can land them together | 15:58 |
| jaosorior | EmilienM: Suire | 15:58 |
| jaosorior | *sure | 15:58 |
| *** zoli is now known as zoli|gone | 15:58 | |
| jistr | EmilienM: my guess is that it's extremely likely that it will pass HA too, given that it passed the others, but still i don't feel comfortable landing a red patch if we know the job was supposed to pass, and it's not an emergency, e.g. a CI fix | 15:58 |
| *** tosky has quit IRC | 15:59 | |
| *** zoli|gone is now known as zoli_gone-proxy | 15:59 | |
| *** aufi has joined #tripleo | 16:03 | |
| openstackgerrit | Emilien Macchi proposed openstack/instack-undercloud: Cleanup Puppet code that deploys Ironic https://review.openstack.org/315238 | 16:04 |
| *** julim has quit IRC | 16:04 | |
| *** egafford has joined #tripleo | 16:05 | |
| EmilienM | jistr: no problem! let's wait recheck. | 16:05 |
| *** julim has joined #tripleo | 16:06 | |
| *** fragatina has quit IRC | 16:07 | |
| *** fragatina has joined #tripleo | 16:08 | |
| *** mikelk has quit IRC | 16:10 | |
| *** ooolpbot has joined #tripleo | 16:11 | |
| ooolpbot | URGENT TRIPLEO TASKS NEED ATTENTION | 16:11 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1582115 | 16:11 |
| *** ooolpbot has quit IRC | 16:11 | |
| openstack | Launchpad bug 1582115 in tripleo "Periodic jobs fail, because there is no instack.qcow2 image" [Critical,Triaged] - Assigned to Derek Higgins (derekh) | 16:11 |
| *** xinwu has joined #tripleo | 16:14 | |
| openstackgerrit | Paul Belanger proposed openstack-infra/tripleo-ci: Update README https://review.openstack.org/316973 | 16:17 |
| *** mcornea has quit IRC | 16:17 | |
| *** spredzy has quit IRC | 16:19 | |
| *** julim has quit IRC | 16:23 | |
| *** spredzy has joined #tripleo | 16:24 | |
| trown | jaosorior: is there a reason you removed +A from https://review.openstack.org/#/c/316361 | 16:24 |
| jaosorior | trown: Cause the dependency hadn't merged. | 16:26 |
| trown | jaosorior: oh ok... it has depends-on though, so should be fine | 16:27 |
| derekh | The DIB revert is gonna fail the gate again, trusty job this time https://review.openstack.org/#/c/316789/ keep the reverifiy's up going | 16:27 |
| derekh | slagle: trown ^ | 16:28 |
| * derekh is outta here | 16:28 | |
| trown | derekh: ack | 16:28 |
| *** derekh has quit IRC | 16:28 | |
| *** Guest28905 has quit IRC | 16:33 | |
| *** julim has joined #tripleo | 16:34 | |
| *** olap has quit IRC | 16:35 | |
| *** mgould has quit IRC | 16:37 | |
| pabelanger | Question, what is the purpose of installing devstack for triplo-ci? trying to understand what is being tested | 16:39 |
| *** aufi has quit IRC | 16:42 | |
| *** tserong has joined #tripleo | 16:43 | |
| *** trown is now known as trown|lunch | 16:45 | |
| *** links has quit IRC | 16:47 | |
| EmilienM | maybe are we reusing some bits from devstack-gate? | 16:47 |
| *** panda has quit IRC | 16:48 | |
| *** panda has joined #tripleo | 16:48 | |
| *** diazjf has quit IRC | 16:53 | |
| openstackgerrit | Paul Belanger proposed openstack-infra/tripleo-ci: Revert check for centos to free HDD space https://review.openstack.org/316973 | 16:56 |
| *** mgould has joined #tripleo | 16:57 | |
| *** shivrao has joined #tripleo | 17:00 | |
| bnemec | pabelanger: Yeah, I think we're just using devstack-gate to set up the source repos. We don't actually run devstack at all. | 17:02 |
| *** fragatina has quit IRC | 17:02 | |
| *** jistr has quit IRC | 17:02 | |
| pabelanger | bnemec: Hmm, seems like a lot of over head just to install bits. Do you know what specifically? | 17:02 |
| pabelanger | adds about 8GB of extra data into the HDD, that I'd like to see about pruning | 17:03 |
| *** ohamada_ has joined #tripleo | 17:05 | |
| *** ohamada__ has quit IRC | 17:05 | |
| bnemec | pabelanger: I'm not sure. This might be a derekh question. | 17:07 |
| pabelanger | bnemec: okay, I'll fire a question to ML too | 17:07 |
| *** ooolpbot has joined #tripleo | 17:10 | |
| ooolpbot | URGENT TRIPLEO TASKS NEED ATTENTION | 17:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1582115 | 17:10 |
| *** ooolpbot has quit IRC | 17:10 | |
| openstack | Launchpad bug 1582115 in tripleo "Periodic jobs fail, because there is no instack.qcow2 image" [Critical,Triaged] - Assigned to Derek Higgins (derekh) | 17:10 |
| *** coolsvap has quit IRC | 17:11 | |
| *** ohamada_ has quit IRC | 17:11 | |
| *** ohamada_ has joined #tripleo | 17:11 | |
| *** david-lyle_ has quit IRC | 17:12 | |
| *** david-lyle has joined #tripleo | 17:13 | |
| *** dsariel has quit IRC | 17:16 | |
| EmilienM | qemu-system-x86_64: -drive file=/var/lib/libvirt/images/seed_3.qcow2,if=none,id=drive-sata0-0-0,format=qcow2,cache=unsafe: could not open disk image /var/lib/libvirt/images/seed_3.qcow2: not in qcow2 format | 17:17 |
| EmilienM | I see that a lot | 17:17 |
| EmilienM | and not in periodic jobs | 17:17 |
| pabelanger | okay, email sent | 17:17 |
| pabelanger | EmilienM: qcow2 should only be used on centos7 | 17:17 |
| pabelanger | EmilienM: wait, what I said is not correct | 17:18 |
| pabelanger | EmilienM: have a log file? | 17:18 |
| EmilienM | pabelanger: maybe it's related to https://review.openstack.org/316789 | 17:19 |
| EmilienM | logs http://logs.openstack.org/38/315238/6/check-tripleo/gate-tripleo-ci-f22-nonha/e8d88f3/console.html#_2016-05-16_16_23_48_771 | 17:20 |
| *** ohamada_ has quit IRC | 17:21 | |
| pabelanger | EmilienM: Seems the job is trying to build instack.qcow2, not sure why it doesn't download it | 17:21 |
| pabelanger | http://logs.openstack.org/38/315238/6/check-tripleo/gate-tripleo-ci-f22-nonha/e8d88f3/console.html#_2016-05-16_16_18_12_718 | 17:21 |
| bnemec | pabelanger: EmilienM: Some of the jobs can't use the cached image because the projects are involved in creating the image, so we need to make sure the change doesn't break it. | 17:22 |
| bnemec | So it's probably all related to the alert bug. | 17:23 |
| pabelanger | I'd enable DIB_DEBUG_TRACE=1 to see what is happening | 17:23 |
| bnemec | We have a revert proposed that should fix the problem: https://review.openstack.org/#/c/316789/ | 17:24 |
| *** coolsvap has joined #tripleo | 17:25 | |
| openstackgerrit | Emilien Macchi proposed openstack/puppet-tripleo: Composable role for RabbitMQ https://review.openstack.org/311849 | 17:29 |
| *** jcoufal has quit IRC | 17:29 | |
| *** coolsvap_ has joined #tripleo | 17:30 | |
| openstackgerrit | Emilien Macchi proposed openstack/tripleo-heat-templates: Deploy RabbitMQ as a composable role https://review.openstack.org/311863 | 17:31 |
| *** coolsvap has quit IRC | 17:34 | |
| *** coolsvap_ is now known as coolsvap | 17:35 | |
| *** ppradhan has joined #tripleo | 17:36 | |
| *** electrofelix has quit IRC | 17:38 | |
| *** xinwu_ has joined #tripleo | 17:43 | |
| *** tserong has quit IRC | 17:43 | |
| *** xinwu has quit IRC | 17:45 | |
| *** xinwu_ is now known as xinwu | 17:45 | |
| *** trown|lunch is now known as trown | 17:46 | |
| *** rcernin has quit IRC | 17:47 | |
| greghaynes | bnemec: any idea what causes the break? | 17:52 |
| greghaynes | bnemec: My best guess is something was the bug isnt in that patch | 17:52 |
| greghaynes | (I am still fine with a revert, however) | 17:52 |
| greghaynes | but that patch fixes cases where failures previously werent being caught and instead would silently succeed | 17:53 |
| greghaynes | so its likely there is an existing failure that is now being caught | 17:53 |
| *** tserong has joined #tripleo | 17:54 | |
| greghaynes | So, can someone more familiar with tripleo dig in to what the root cause is because that patch fixes a pretty bad dib bug so I'd like it to go in | 17:54 |
| bnemec | greghaynes: I don't know, I haven't looked into it that closely. | 17:54 |
| greghaynes | I am also more than happy to help debug :) I am probably not the most effective debugger of the tripleo build not succeeding though | 17:55 |
| openstackgerrit | Ben Nemec proposed openstack/instack-undercloud: Switch to nova v2.1 and remove v3 https://review.openstack.org/317033 | 17:57 |
| greghaynes | alternatively, I can post that patch as a major version change and youall can try pinning dib... | 17:57 |
| bnemec | It's a pretty simple image build that is failing: http://git.openstack.org/cgit/openstack-infra/tripleo-ci/tree/toci_instack.sh#n210 | 17:59 |
| bnemec | Only two elements and a bunch of packages. | 18:00 |
| greghaynes | Is there a log of the failure somewhere? | 18:00 |
| bnemec | greghaynes: http://logs.openstack.org/38/315238/6/check-tripleo/gate-tripleo-ci-f22-nonha/e8d88f3/logs/instack-build.txt.gz | 18:01 |
| greghaynes | ty | 18:01 |
| bnemec | EmilienM: For the nova v2.1 change, we'll need https://review.openstack.org/317033 and a similar patch for t-h-t before we'll know if puppet-keystone will update the endpoints correctly. We don't use the defaults from puppet-nova. | 18:02 |
| bnemec | Oh, actually that already got changed in tht. | 18:05 |
| bnemec | Except we left the tenant_id in the 2.1 endpoint. | 18:06 |
| *** fragatina has joined #tripleo | 18:06 | |
| *** dtantsur is now known as dtantsur|afk | 18:06 | |
| *** lucasagomes is now known as lucas-dinner | 18:06 | |
| *** julim has quit IRC | 18:07 | |
| *** julim has joined #tripleo | 18:08 | |
| openstackgerrit | Ben Nemec proposed openstack/tripleo-heat-templates: Remove tenant_id from nova v2.1 endpoint https://review.openstack.org/317036 | 18:08 |
| adarazs | trown: hi! I'm using the latest promoted master delorean and it dies during the beginning of the ceph node deployment (it's a 1 controller, 1 compute, 1 ceph deploy) ... does the promote test ceph? | 18:09 |
| *** ooolpbot has joined #tripleo | 18:10 | |
| ooolpbot | URGENT TRIPLEO TASKS NEED ATTENTION | 18:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1582115 | 18:10 |
| *** ooolpbot has quit IRC | 18:10 | |
| openstack | Launchpad bug 1582115 in tripleo "Periodic jobs fail, because there is no instack.qcow2 image" [Critical,Triaged] - Assigned to Derek Higgins (derekh) | 18:10 |
| *** mgould has quit IRC | 18:10 | |
| *** diazjf has joined #tripleo | 18:10 | |
| trown | adarazs: nope there is not a ceph job in the promote pipeline... | 18:11 |
| adarazs | trown: okay, so that explains why my deployment fails with it :/ | 18:11 |
| adarazs | trown: could you take a look at my env? I went through the usual debugging steps for "no valid host found" error (http://docs.openstack.org/developer/tripleo-docs/troubleshooting/troubleshooting.html#no-valid-host-found-error) and I'm out of ideas why the deployment fails like that :/ | 18:12 |
| trown | adarazs: hmm... it fails with "no valid host"? | 18:14 |
| *** sambetts is now known as sambetts|afk | 18:15 | |
| adarazs | trown: or if you see something in the logs here: https://ci.centos.org/artifacts/rdo/jenkins-tripleo-periodic-cat-delorean-stable-master-4/undercloud/home/stack/overcloud_deploy.log.gz | 18:15 |
| adarazs | trown: yeah. even though the nova flavor-list and the ironic node-show looks okay. and the VM looks okay, and the other two VMs deploy properly consistenly. | 18:15 |
| trown | adarazs: ok, that is likely transient | 18:16 |
| trown | 2016-05-16 04:07:01.057 8693 DEBUG nova.scheduler.filters.ram_filter [req-80856a7c-a14a-4566-a13d-f65a1e8388c8 9154d080648e489d9224d5cee7738b59 4612492946f9464ea0717e73c94ac62c - - -] (undercloud, 1dca9db6-f630-489e-a191-85c97bafb98d) ram: 6144MB disk: 51200MB io_ops: 0 instances: 0 does not have 6144 MB usable ram before overcommit, it only has 0 MB. host_passes /usr/lib/python2.7/site-packages/nova/scheduler/filters/ram_filter.py:45 | 18:17 |
| trown | 2016-05-16 04:07:01.057 8693 INFO nova.filters [req-80856a7c-a14a-4566-a13d-f65a1e8388c8 9154d080648e489d9224d5cee7738b59 4612492946f9464ea0717e73c94ac62c - - -] Filter RamFilter returned 0 hosts | 18:17 |
| trown | https://ci.centos.org/artifacts/rdo/jenkins-tripleo-periodic-cat-delorean-stable-master-4/undercloud/var/log/nova/nova-scheduler.log.gz | 18:17 |
| trown | adarazs: although I dont really understand how we would hit that when we have 1 node per flavor | 18:19 |
| *** jtomasek has quit IRC | 18:19 | |
| adarazs | trown: I did about 4 runs so far, all 4 had this same error o_o | 18:20 |
| adarazs | with only the ceph node. | 18:20 |
| *** shardy has quit IRC | 18:24 | |
| trown | adarazs: but it is actually the compute that is failing to deploy... and it is failing at ironic, so I dont think ceph is related | 18:25 |
| *** yamahata has quit IRC | 18:26 | |
| *** yamahata has joined #tripleo | 18:27 | |
| trown | adarazs: hmm, looking at https://ci.centos.org/artifacts/rdo/jenkins-tripleo-periodic-cat-delorean-stable-master-4/undercloud/home/stack/overcloud_deploy.log.gz I think you are missing the storage environment for ceph also | 18:27 |
| adarazs | hmmm, I could swear it was ceph the last time I reproduced | 18:27 |
| trown | -e /usr/share/openstack-tripleo-heat-templates/environments/storage-environment.yaml | 18:27 |
| *** fragatin_ has joined #tripleo | 18:28 | |
| adarazs | trown: you're right. okay, that got me out the dead end. thank you for looking! ^_^ | 18:28 |
| trown | np | 18:29 |
| *** fragatina has quit IRC | 18:32 | |
| *** coolsvap is now known as coolsvap|zzzzzzz | 18:36 | |
| *** penick has joined #tripleo | 18:39 | |
| *** hogepodge has joined #tripleo | 18:44 | |
| EmilienM | bnemec: ok I'll look after dinner | 18:53 |
| EmilienM | oh you did, ince | 18:53 |
| EmilienM | nice* | 18:53 |
| *** morazi has quit IRC | 18:55 | |
| larsks | trown: have you seen this error before? "The files ('overcloud-without-mergepy.yaml', 'overcloud.yaml') not found in the /usr/share/openstack-tripleo-heat-templates/ directory" | 18:58 |
| larsks | trown: (the files appear to be there despite the error message) | 18:58 |
| larsks | getting that with a quickstart deploy with network_isolation: false. | 18:59 |
| trown | larsks: the error is bad... it means *some* environment file was not found | 18:59 |
| larsks | trown: I see. That makes a lot more sense. | 19:00 |
| trown | https://review.openstack.org/#/c/300462/ will fix it | 19:00 |
| larsks | That would appear to be because we are passing -e /home/stack/network-environment.yaml when network_isolation is false (so we don't generate network-environment.yaml) | 19:01 |
| trown | wow that needs some love | 19:01 |
| larsks | Yeah. As far as I can tell, our network configuration stuff has been all messed up for a while, and only works if you happen to use network_isolation: true with default values. | 19:01 |
| *** noslzzp has quit IRC | 19:02 | |
| trown | I was meaning the review to fix that error message, but ya | 19:03 |
| trown | larsks: there is a config for no net iso: https://github.com/openstack/tripleo-quickstart/blob/master/playbooks/centosci/minimal_no_netiso.yml | 19:03 |
| *** noslzzp has joined #tripleo | 19:03 | |
| larsks | trown: ...yeah, so, that would seem to be representative of the problem. | 19:04 |
| larsks | Whether or not we generate network-environment.yaml is indepent of whether or not we ask tripleo to use which is independent of the filename we provide on the deploy command line. | 19:04 |
| * larsks headdesks | 19:05 | |
| trown | larsks: ya, I think https://github.com/openstack/tripleo-quickstart/blob/master/playbooks/roles/common/defaults/main.yml#L102 should default to false, and we should set it true in the minimal config | 19:06 |
| larsks | trown: but false or true, it should never requiring a matching explicit "-e ~/network-environment.yaml" in extra_args. | 19:06 |
| larsks | That should happen implicitly if network_isolation=true. | 19:07 |
| trown | +1 | 19:07 |
| trown | it needs all three environment files though: https://github.com/openstack/tripleo-quickstart/blob/master/playbooks/centosci/minimal.yml#L17-L19 | 19:08 |
| trown | but 100% agree that network_isolation=true should take care of all of that | 19:08 |
| larsks | Sure, but the other two we're not generating as part of the quickstart. | 19:08 |
| larsks | But let me see what I can do. | 19:08 |
| *** pblaho has joined #tripleo | 19:09 | |
| *** ooolpbot has joined #tripleo | 19:10 | |
| ooolpbot | URGENT TRIPLEO TASKS NEED ATTENTION | 19:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1582115 | 19:10 |
| *** ooolpbot has quit IRC | 19:10 | |
| openstack | Launchpad bug 1582115 in tripleo "Periodic jobs fail, because there is no instack.qcow2 image" [Critical,Triaged] - Assigned to Derek Higgins (derekh) | 19:10 |
| *** fragatin_ has quit IRC | 19:23 | |
| *** fragatina has joined #tripleo | 19:23 | |
| openstackgerrit | Paul Belanger proposed openstack-infra/tripleo-ci: Build and upload instack.raw image https://review.openstack.org/315072 | 19:24 |
| openstackgerrit | Sagi Shnaidman proposed openstack-infra/tripleo-ci: WIP: DONT MERGE Running tests https://review.openstack.org/316436 | 19:37 |
| *** lucas-dinner has quit IRC | 19:38 | |
| *** rcernin has joined #tripleo | 19:42 | |
| *** lucasagomes has joined #tripleo | 19:43 | |
| pabelanger | I'd like to help landing https://review.openstack.org/#/c/316973/ | 19:46 |
| pabelanger | that fixes centos-7 HDD space issue | 19:46 |
| pabelanger | we bumped to 25GB, but still running out of room. So we are reverting our rm -rf /opt logic | 19:46 |
| EmilienM | pabelanger: why can't we only keep what we need? | 19:47 |
| EmilienM | don't we know what we actually need? | 19:47 |
| pabelanger | EmilienM: I'd rather figure out what devstack is doing, since that use 10GB+ | 19:47 |
| openstackgerrit | Ben Nemec proposed openstack/instack-undercloud: Switch to nova v2.1 and remove v3 https://review.openstack.org/317033 | 19:47 |
| *** rohitpagedar__ has joined #tripleo | 19:47 | |
| pabelanger | EmilienM: I suspect a lot of this can be now moved into bindep | 19:47 |
| EmilienM | what??? | 19:48 |
| EmilienM | 10GB? | 19:48 |
| *** xinwu has quit IRC | 19:48 | |
| pabelanger | yup | 19:48 |
| pabelanger | we boot with 7.5GB today | 19:48 |
| EmilienM | pabelanger: +2 | 19:48 |
| pabelanger | after devstack runs, we are up to 17.5GB | 19:48 |
| *** yamahata has quit IRC | 19:48 | |
| pabelanger | once we get instack raw images, we can them remove some code in toci_gate.sh too | 19:48 |
| *** tserong has quit IRC | 19:49 | |
| pabelanger | I'd like to keep /opt/git, since it does help with zuul-cloner | 19:49 |
| *** pblaho has quit IRC | 19:49 | |
| *** dtrainor has quit IRC | 19:49 | |
| larsks | trown: of the other settings in https://github.com/openstack/tripleo-quickstart/blob/master/playbooks/centosci/minimal.yml#L17-L19, do neutron-network-type and neutron-tunnel-types have sane defaults? Or should network_isolation: true also imply those? | 19:50 |
| greghaynes | bnemec: about that revert - https://review.openstack.org/#/c/192521/ would be a big help in preventing future breakages to that image | 19:50 |
| *** lucasagomes has quit IRC | 19:51 | |
| trown | larsks: they have sane defaults, but we still probably want the minimal config to do vxlan since we care about CI'ing that more | 19:53 |
| larsks | Sure. My question is, in general, if someone were to set network_isolation: true without specifying vxlan networking, would we expect it to Just Work? | 19:53 |
| trown | larsks: yep | 19:53 |
| larsks | Awesome. Thanks! | 19:53 |
| *** mbound has joined #tripleo | 19:54 | |
| *** jaosorior has quit IRC | 19:56 | |
| *** lucasagomes has joined #tripleo | 19:57 | |
| *** tserong has joined #tripleo | 20:00 | |
| *** dtrainor has joined #tripleo | 20:01 | |
| *** oshvartz has quit IRC | 20:04 | |
| *** oshvartz has joined #tripleo | 20:05 | |
| *** oshvartz has quit IRC | 20:05 | |
| *** oshvartz has joined #tripleo | 20:06 | |
| *** paramite has quit IRC | 20:09 | |
| *** ooolpbot has joined #tripleo | 20:10 | |
| ooolpbot | URGENT TRIPLEO TASKS NEED ATTENTION | 20:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1582115 | 20:10 |
| *** ooolpbot has quit IRC | 20:10 | |
| openstack | Launchpad bug 1582115 in tripleo "Periodic jobs fail, because there is no instack.qcow2 image" [Critical,Triaged] - Assigned to Derek Higgins (derekh) | 20:10 |
| *** yamahata has joined #tripleo | 20:12 | |
| *** dmk0202 has joined #tripleo | 20:14 | |
| trown | larsks: would you mind looking at https://review.openstack.org/#/c/309995/ when you get a chance? I have never used sshuttle | 20:16 |
| larsks | Yeah, so, the issue there is that I think it's wrong, depending on whether or not network_isolation is enabled :). | 20:17 |
| trown | lol :) | 20:18 |
| larsks | I have that on my "list of things to look at once I have fixed networking" | 20:18 |
| trown | perfect, thanks | 20:18 |
| larsks | Also, the HostsEntry output from the overcloud stack is also problematic with network isolation enabled. | 20:18 |
| larsks | Which doesn't have an operational impact but may impact our docs. | 20:18 |
| openstackgerrit | Merged openstack/tripleo-quickstart: add the option to specify an alternative requirements file https://review.openstack.org/313646 | 20:19 |
| openstackgerrit | Merged openstack/tripleo-quickstart: use nthhost() instead of ipaddr('address') filter https://review.openstack.org/316185 | 20:20 |
| *** bvandenh has quit IRC | 20:22 | |
| weshay | wonder if it's worth allowing a retry on the undercloud install.. I just had newton master stable fail the first time and work upon one retry | 20:23 |
| weshay | in quickstart | 20:23 |
| trown | weshay: what failed the first time? | 20:24 |
| EmilienM | yeah, what failed? | 20:25 |
| openstackgerrit | Ben Nemec proposed openstack/instack-undercloud: Generate most of the pystache context automatically https://review.openstack.org/297319 | 20:25 |
| EmilienM | if it's puppet run, that's a bug we need to fix | 20:25 |
| *** jayg is now known as jayg|g0n3 | 20:25 | |
| weshay | trown, neutron service starting | 20:25 |
| weshay | sry.. dbsync | 20:26 |
| weshay | ESC[1;31mError: /Stage[main]/Neutron::Db::Sync/Exec[neutron-db-sync]: Command exceeded timeout | 20:26 |
| weshay | [2016-05-16 19:47:07,590] (os-refresh-config) [ERROR] during configure phase. [Command '['dib-run-parts', '/usr/libexec/os-refresh-config/configure.d']' returned non-zero exit status 6] | 20:26 |
| EmilienM | that's a bug | 20:27 |
| EmilienM | can you give /var/log/neutron/*.log? | 20:27 |
| EmilienM | it seems like maybe mysql was not ready | 20:27 |
| weshay | sure.. sec | 20:27 |
| weshay | trown, so.. basically this was promoted as stable, but failed due to timing issues .. | 20:29 |
| EmilienM | trown: can we +A https://review.openstack.org/#/c/316360/ and https://review.openstack.org/#/c/316361/ ? everything is green now | 20:29 |
| *** jprovazn has quit IRC | 20:29 | |
| *** hjensas has joined #tripleo | 20:30 | |
| *** hjensas has quit IRC | 20:30 | |
| *** hjensas has joined #tripleo | 20:30 | |
| trown | EmilienM: done | 20:30 |
| EmilienM | trown: ++ | 20:30 |
| *** dtrainor has quit IRC | 20:31 | |
| EmilienM | thanks | 20:31 |
| larsks | Has anyone seen this before? Ipxe is attempting to fetch boot.ipxe from the undercloud. The client sends a SYN to the undercloud, the undercloudn sends a SYN to the client, and the client issues a RST. https://gist.github.com/larsks/8b878fb923a5377e30a0f528ca5ec688 | 20:32 |
| pabelanger | Do we have a resource map of what the tripleo cloud looks like today? | 20:34 |
| *** eggmaster has quit IRC | 20:34 | |
| openstackgerrit | Merged openstack/tripleo-heat-templates: Remove Nova EC2 deployment https://review.openstack.org/316360 | 20:36 |
| *** dtrainor has joined #tripleo | 20:40 | |
| *** xinwu has joined #tripleo | 20:41 | |
| *** liverpooler has quit IRC | 20:42 | |
| *** liverpooler has joined #tripleo | 20:46 | |
| openstackgerrit | Ben Nemec proposed openstack-infra/tripleo-ci: Explode log tars so they can be indexed https://review.openstack.org/314272 | 20:52 |
| openstackgerrit | Ben Nemec proposed openstack-infra/tripleo-ci: Record Heat resource deployment times to Graphite https://review.openstack.org/314308 | 20:52 |
| *** julim has quit IRC | 20:55 | |
| *** dtrainor has quit IRC | 20:58 | |
| openstackgerrit | Merged openstack/puppet-tripleo: Remove Nova EC2 HAproxy endpoint https://review.openstack.org/316361 | 21:00 |
| EmilienM | weshay: ok I looked the logs | 21:02 |
| EmilienM | and I even could reproduce something similar | 21:02 |
| *** trown is now known as trown|outtypewww | 21:03 | |
| EmilienM | weshay: I'll look tomorrow, it's 11pm where i am now | 21:03 |
| weshay | oh.. k.. I'm trying a clean install atm.. | 21:03 |
| weshay | EmilienM, oh man.. go to sleep man | 21:03 |
| EmilienM | dmsimard: damn, also hitting the paramiko issue in passed-ci | 21:04 |
| EmilienM | we need this update asap, 10% of our CI jobs are failing | 21:04 |
| weshay | this is surely not a consistent issue | 21:04 |
| weshay | k | 21:04 |
| EmilienM | weshay: no, I had it for nova also | 21:05 |
| EmilienM | I'll continue tomorrow | 21:05 |
| weshay | rockin | 21:05 |
| *** ooolpbot has joined #tripleo | 21:10 | |
| ooolpbot | URGENT TRIPLEO TASKS NEED ATTENTION | 21:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1582115 | 21:10 |
| *** ooolpbot has quit IRC | 21:10 | |
| openstack | Launchpad bug 1582115 in tripleo "Periodic jobs fail, because there is no instack.qcow2 image" [Critical,Triaged] - Assigned to Derek Higgins (derekh) | 21:10 |
| larsks | Anybody around who might be able to help diagnose some pxe booting issues? | 21:11 |
| larsks | Clients and undercloud appear to be able to talk to each other, but clients are unable to fetch boot.ipxe... | 21:11 |
| *** dtrainor has joined #tripleo | 21:11 | |
| *** rhallisey has quit IRC | 21:13 | |
| *** lblanchard has quit IRC | 21:15 | |
| *** fultonj has quit IRC | 21:15 | |
| larsks | Oooooh, never mind, I figured it out. | 21:17 |
| larsks | THere was an errant undercloud vm hanging around with a duplicate ip. | 21:17 |
| *** weshay has quit IRC | 21:19 | |
| ianw | sorry about the dib error thing breaking tripleo, mea culpa | 21:27 |
| *** dtrainor has quit IRC | 21:30 | |
| *** Goneri has quit IRC | 21:33 | |
| openstackgerrit | Merged openstack/diskimage-builder: Add centos7 test https://review.openstack.org/192521 | 21:43 |
| *** MaxPC has quit IRC | 21:50 | |
| openstackgerrit | Merged openstack/diskimage-builder: Fix path issue for locale-archive.tmpl https://review.openstack.org/316831 | 22:01 |
| openstackgerrit | Merged openstack/tripleo-common: Initial Integration of Mistral into TripleO https://review.openstack.org/282366 | 22:04 |
| *** ooolpbot has joined #tripleo | 22:10 | |
| ooolpbot | URGENT TRIPLEO TASKS NEED ATTENTION | 22:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1582115 | 22:10 |
| openstack | Launchpad bug 1582115 in tripleo "Periodic jobs fail, because there is no instack.qcow2 image" [Critical,Triaged] - Assigned to Derek Higgins (derekh) | 22:10 |
| *** ooolpbot has quit IRC | 22:10 | |
| *** dmk0202 has quit IRC | 22:10 | |
| *** diazjf has quit IRC | 22:20 | |
| *** penick has quit IRC | 22:27 | |
| *** xinwu has quit IRC | 22:34 | |
| *** pradk- has quit IRC | 22:46 | |
| openstackgerrit | Merged openstack/diskimage-builder: Revert "Properly fail/trap in eval_run_d" https://review.openstack.org/316789 | 22:46 |
| *** MaxPC has joined #tripleo | 22:46 | |
| *** jobewan has quit IRC | 22:52 | |
| openstackgerrit | Jason E. Rist proposed openstack/tripleo-ui: Refactor notification for better display https://review.openstack.org/316874 | 23:03 |
| openstackgerrit | Ryan Brady proposed openstack/tripleo-common: Add Plan Deletion https://review.openstack.org/299937 | 23:05 |
| *** rcernin has quit IRC | 23:08 | |
| *** ooolpbot has joined #tripleo | 23:10 | |
| ooolpbot | URGENT TRIPLEO TASKS NEED ATTENTION | 23:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1582115 | 23:10 |
| *** ooolpbot has quit IRC | 23:10 | |
| openstack | Launchpad bug 1582115 in tripleo "Periodic jobs fail, because there is no instack.qcow2 image" [Critical,Triaged] - Assigned to Derek Higgins (derekh) | 23:10 |
| openstackgerrit | Jason E. Rist proposed openstack/tripleo-ui: Refactor notification for better display https://review.openstack.org/316874 | 23:10 |
| *** mbound has quit IRC | 23:14 | |
| *** saneax_AFK is now known as saneax | 23:15 | |
| *** rohitpagedar__ has quit IRC | 23:24 | |
| openstackgerrit | Ben Kero proposed openstack/diskimage-builder: Revert "Revert "Properly fail/trap in eval_run_d"" and fix PIPESTATUS https://review.openstack.org/317137 | 23:26 |
| openstackgerrit | Ryan Brady proposed openstack/tripleo-common: Adds Plan List Action https://review.openstack.org/300221 | 23:29 |
| openstackgerrit | Ryan Brady proposed openstack/tripleo-common: Add Heat Capabilities Actions https://review.openstack.org/284460 | 23:31 |
| openstackgerrit | Ryan Brady proposed openstack/tripleo-common: Adds Deployment Action & Workflow https://review.openstack.org/298732 | 23:31 |
| pabelanger | Anybody online that has admin credentials to tripleo cloud? | 23:48 |
| pabelanger | Looking for confirmation about the zone settings | 23:48 |
| *** apetrich_ has quit IRC | 23:53 | |
| *** rdo has quit IRC | 23:53 | |
| *** r-mibu has quit IRC | 23:55 | |
Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!