Wednesday, 2019-03-06

*** tosky has quit IRC00:00
*** dmellado has quit IRC00:03
rdogerritrdo-trunk created rdoinfo master: Promote CBS tags update for pike-release  https://review.rdoproject.org/r/1911800:03
*** wolverineav has quit IRC00:04
*** dmellado has joined #rdo00:04
rdogerritrdo-trunk created rdoinfo master: Promote CBS tags update for queens-release  https://review.rdoproject.org/r/1911900:05
rdogerritrdo-trunk created rdoinfo master: Promote CBS tags update for rocky-release  https://review.rdoproject.org/r/1912000:07
rdogerritrdo-trunk proposed rdoinfo master: Bump rdoinfo stein-uc tags to latest upper-constraints  https://review.rdoproject.org/r/1896500:09
*** ttsiouts has quit IRC00:11
*** imcleod has joined #rdo00:13
*** wolverineav has joined #rdo00:22
*** sshnaidm is now known as sshnaidm|afk00:28
*** artom has joined #rdo00:42
*** wolverineav has quit IRC00:48
*** wolverineav has joined #rdo00:51
*** wolverineav has quit IRC01:03
*** wolverineav has joined #rdo01:04
*** wolverineav has quit IRC01:09
*** oanson has quit IRC01:13
*** chem has quit IRC01:25
*** chem has joined #rdo01:25
*** sdake has quit IRC01:29
*** wolverineav has joined #rdo01:36
*** sdake has joined #rdo01:47
*** wolverineav has quit IRC02:00
*** wolverineav has joined #rdo02:00
*** sdake has quit IRC02:05
*** Son_Goku has quit IRC02:06
*** imcleod has quit IRC02:11
*** dsneddon has quit IRC02:21
*** wolverineav has quit IRC02:22
*** Son_Goku has joined #rdo02:23
*** chem has quit IRC02:25
*** chem has joined #rdo02:26
*** wolverineav has joined #rdo02:27
*** sdake has joined #rdo02:30
*** wolverineav has quit IRC02:40
*** sdake has quit IRC02:42
*** Son_Goku has quit IRC02:43
*** dsneddon has joined #rdo02:46
*** Son_Goku has joined #rdo02:50
*** imcleod has joined #rdo02:52
*** ykarel has joined #rdo02:54
*** Son_Goku has quit IRC02:54
*** dsneddon has quit IRC02:57
*** Son_Goku has joined #rdo03:01
*** Son_Goku has quit IRC03:01
rdogerritMerged rdoinfo master: Promote CBS tags update for queens-release  https://review.rdoproject.org/r/1911903:02
rdogerritMerged rdoinfo master: Promote CBS tags update for pike-release  https://review.rdoproject.org/r/1911803:03
rdogerritMerged rdoinfo master: Promote CBS tags update for rocky-release  https://review.rdoproject.org/r/1912003:03
rdogerritYatin Karel proposed rdo-infra/ansible-role-weirdo-puppet-openstack master: Exclude git_resource from the list of puppet modules to install from rpm  https://review.rdoproject.org/r/1907703:04
*** rlandy|bbl is now known as rlandy03:06
*** dsneddon has joined #rdo03:07
rdogerritMerged rdoinfo master: Bump python-etcd3gw in stein-candidate  https://review.rdoproject.org/r/1911503:08
*** imcleod has quit IRC03:08
*** dsneddon has quit IRC03:13
*** Son_Goku has joined #rdo03:16
*** sdake has joined #rdo03:21
*** sdake has quit IRC03:23
*** Son_Goku has quit IRC03:24
*** sdake_ has joined #rdo03:24
*** dsneddon has joined #rdo03:25
*** chem has quit IRC03:26
*** chem has joined #rdo03:27
*** imcleod has joined #rdo03:27
*** dsneddon has quit IRC03:32
*** imcleod has quit IRC03:33
*** Son_Goku has joined #rdo03:35
*** Son_Goku has quit IRC03:36
rdogerritwes hayutin created config master: fix indentation on env  https://review.rdoproject.org/r/1912103:38
rdogerritMerged config master: fix indentation on env  https://review.rdoproject.org/r/1912103:43
*** wolverineav has joined #rdo03:43
*** dsneddon has joined #rdo03:44
*** wolverineav has quit IRC03:44
rdogerritYatin Karel proposed rdo-jobs master: [DNM] Rerun standalone job to test config change  https://review.rdoproject.org/r/1901703:45
ykarelrlandy, weshay|rover testing if no other issue in config before next peridic run ^^03:46
weshay|roverykarel++03:47
rlandyfor centos no other error - for fedora there is the mirror problem03:47
ykarelwhat mirror?03:47
weshay|rovercurl --silent http://mirror.regionone.rdo-cloud-tripleo.rdoproject.org:8080/rdo/fedora28/71/d9/71d9a66abe93049a91d433f805c045abe135303a_e6dde0f8/delorean.repo -S03:48
weshay|roverwas failing in the jobs03:48
weshay|roverhttps://logs.rdoproject.org/openstack-periodic/git.openstack.org/openstack-infra/tripleo-ci/master/periodic-tripleo-ci-fedora-28-standalone-master/5e38cbb/logs/undercloud/home/zuul/repo_setup.log.txt.gz#_2019-03-05_23_19_2203:48
weshay|roverbut seems to work fine via cli now03:48
ykarelweshay|rover, possible network glich when the job ran03:48
ykarelmirror looks correct03:48
ykareldid all fedora jobs failed with same error?03:48
weshay|rovermaybe it's the cut command rlandy03:49
*** janki has joined #rdo03:49
*** janki has quit IRC03:50
*** janki has joined #rdo03:50
weshay|roverhrm.. nope03:52
weshay|rovercurl --silent http://mirror.regionone.rdo-cloud-tripleo.rdoproject.org:8080/rdo/fedora28/71/d9/71d9a66abe93049a91d433f805c045abe135303a_e6dde0f8/delorean.repo -S | grep baseurl |cut -d= -f203:52
rlandyweshay|rover: it does resolve now03:52
weshay|roverhttps://trunk.rdoproject.org/fedora/71/d9/71d9a66abe93049a91d433f805c045abe135303a_e6dde0f803:52
weshay|roveris that I get03:52
weshay|roverthat should work03:52
rlandysmae03:53
rlandysame03:53
rlandyshould work03:53
weshay|roverwell.. we'll see what we get tomorrow I guess03:53
*** dsneddon has quit IRC03:53
rlandyweshay|rover: well that is progress03:53
weshay|roverrlandy it's odd that the fedora build job worked03:56
weshay|roverit should be using the same mirrors03:56
weshay|roverbut I'm sure is configured differently03:56
weshay|roverrlandy https://logs.rdoproject.org/openstack-periodic/git.openstack.org/openstack-infra/tripleo-ci/master/periodic-tripleo-fedora-28-master-containers-build-push/09a6978/etc/yum.repos.d/fedora.repo.txt.gz03:57
weshay|roverrlandy maybe we do change it to fedora03:57
weshay|rovervs fedora2803:57
rlandyhmmm03:57
rlandytesting03:57
*** sdake_ has quit IRC03:58
rlandyfedora28 works fine as well though03:58
rlandysame deal03:58
weshay|roverhttps://logs.rdoproject.org/openstack-periodic/git.openstack.org/openstack-infra/tripleo-ci/master/periodic-tripleo-ci-fedora-28-standalone-master/5e38cbb/logs/undercloud/etc/yum.repos.d/fedora.repo.txt.gz03:58
weshay|roverwhat the?03:58
*** wolverineav has joined #rdo03:58
weshay|roverya.. might as well03:58
rlandyyou want the review back?03:59
weshay|roverI think so03:59
weshay|roverhttps://review.openstack.org/#/c/641187/103:59
rlandywell - let's see what fails in the next promotion04:00
weshay|roverrlandy https://logs.rdoproject.org/openstack-periodic/git.openstack.org/openstack-infra/tripleo-ci/master/periodic-tripleo-fedora-28-master-containers-build-push/09a6978/etc/yum.repos.d/delorean.repo.txt.gz04:00
rlandywe won't have this through the gate before that kicks04:00
*** dsneddon has joined #rdo04:00
ykarelrlandy, weshay|rover both fedora and fedora28 are same04:00
ykarelsame repo04:00
rlandyboth resolve04:00
weshay|roverykarel I know.. but what about the proxy04:00
weshay|roverclearly fedora is working where fedora28 did not04:00
weshay|roverit would have to recache everything04:00
ykarellocally also?04:01
rlandywell the same curl works now04:01
weshay|roverlocally it works04:01
weshay|roverI would assume a caching proxy would cache the two dirs seperately04:01
ykarelyes possibly network issue when job ran, so both fedora/fedora28 either pass or fail04:01
weshay|roverwhy would we awant that04:01
weshay|rovermaybe04:01
weshay|roverstill04:01
rlandyto match centos7 naming04:01
weshay|roverlet's just be consistent04:01
weshay|roverthere is never going to be a fedora 29 job04:02
weshay|rovernever04:02
rlandyidk04:02
ykarelweshay|rover, same symlink:- https://github.com/rdo-infra/puppet-dlrn/blob/master/examples/common.yaml#L18504:02
rlandyeither way the next promotion will run w/o this change04:02
rlandylet's see what happens04:02
ykarelfedora28 was created after tripleoCI team request to be namming with distro+version04:03
rlandyack - I remember that request04:03
rlandyrfolco|pto put it in I think04:03
ykarelmay be04:04
*** dsneddon has quit IRC04:05
*** rlandy has quit IRC04:12
*** ramishra has joined #rdo04:14
*** imcleod has joined #rdo04:16
*** chem has quit IRC04:27
*** chem has joined #rdo04:27
*** imcleod has quit IRC04:35
*** dsneddon has joined #rdo04:35
*** dsneddon has quit IRC04:40
*** sdake has joined #rdo04:49
*** sdake has quit IRC04:49
*** dsneddon has joined #rdo04:54
*** sdake has joined #rdo04:55
*** raukadah is now known as chandankumar04:56
rdogerritYatin Karel created config master: Add DLRANI_USERNAME in quoutes  https://review.rdoproject.org/r/1912204:58
*** dsneddon has quit IRC04:59
ykarelweshay|rover, trying ^^ next periodic run in 10 minutes04:59
ykarelchandankumar, let's try ^^04:59
rdogerritMerged config master: Add DLRANI_USERNAME in quoutes  https://review.rdoproject.org/r/1912205:02
*** sdake has joined #rdo05:26
*** sdake has quit IRC05:26
*** dsneddon has joined #rdo05:27
rdogerritYatin Karel proposed rdoinfo master: CBS tags sync with uc for stein-candidate  https://review.rdoproject.org/r/1690405:30
*** dsneddon has quit IRC05:32
*** sdake has joined #rdo05:38
*** wolverineav has quit IRC05:40
*** dsneddon has joined #rdo06:04
*** udesale has joined #rdo06:07
*** dsneddon has quit IRC06:09
*** gkadam has quit IRC06:13
*** wolverineav has joined #rdo06:14
*** wolverineav has quit IRC06:18
*** wolverineav has joined #rdo06:18
*** chem has quit IRC06:28
*** chem has joined #rdo06:29
*** wolverineav has quit IRC06:32
*** baha has joined #rdo06:42
*** jfrancoa has joined #rdo06:52
*** quiquell|off is now known as quiquell06:53
*** ccamposr has joined #rdo06:59
*** mosulica has joined #rdo07:00
*** dsneddon has joined #rdo07:04
*** dsneddon has quit IRC07:08
*** aufi has joined #rdo07:12
*** paramite has joined #rdo07:15
*** dsneddon has joined #rdo07:20
*** dsneddon has quit IRC07:25
*** chem has quit IRC07:29
*** ksambor has joined #rdo07:30
*** chem has joined #rdo07:32
*** rascasoft has quit IRC07:36
*** rascasoft has joined #rdo07:37
*** ccamacho has joined #rdo07:41
*** dsneddon has joined #rdo07:44
*** dsneddon has quit IRC07:48
*** quiquell is now known as quiquell|brb07:50
*** dsneddon has joined #rdo07:50
*** jlibosva has joined #rdo07:55
*** hseipp has joined #rdo08:01
*** rpittau|afk is now known as rpittau08:02
*** dtantsur|afk is now known as dtantsur08:03
*** dtantsur is now known as dtantsur|mtg08:03
*** wolverineav has joined #rdo08:03
*** kopecmartin|off is now known as kopecmartin08:05
*** wolverineav has quit IRC08:08
*** wolverineav has joined #rdo08:10
*** dsneddon has quit IRC08:14
*** gkadam has joined #rdo08:14
*** wolverineav has quit IRC08:17
*** wolverineav has joined #rdo08:18
*** amoralej|off is now known as amoralej08:19
amoralejgood morning08:20
*** dsneddon has joined #rdo08:22
ykarelo/08:23
amoralejykarel, have you noticed anything better in ci jobs?08:25
amoraleji mean, nested virt issue08:25
ykarelamoralej, nope it's still same08:26
ykarelstill seeing failures08:26
amoralejyesterday i tried to disable nested in weirdo08:26
ykarelyou put patch in weirdo too08:26
*** dsneddon has quit IRC08:26
* ykarel looking08:26
amoralejhttps://review.rdoproject.org/r/#/c/19077/08:27
amoralejmmm08:27
amoralejdidn't work08:27
ykarelyes, nested is enabled in weirdo08:27
amoralejbtw ocata job is consistenly timing out, i'm not sure if it's the same issue08:27
amoralejykarel, and py3 job is always passing08:27
*** quiquell|brb is now known as quiquell08:28
amoralejmm08:28
ykarelamoralej, py3 is fedora:- https://review.rdoproject.org/r/#/c/18514/2/playbooks/puppet-openstack-scenario002.yml08:28
amoralejwe disabled nested in fedora08:28
amoraleji remember08:28
ykarelamoralej, yes ^^08:28
ykareltemporary we can change ^^ to disable nested in centos as well until it's sorted out in rdo-cloud08:29
ykareland make your puppet-openstack patch depends on weirdo patch08:29
amoralejoh08:29
*** tosky has joined #rdo08:29
*** mosulica has quit IRC08:29
amoralejdidn't i send a new patch set?08:29
*** pcaruana has joined #rdo08:29
ykarelamoralej, you sent, i commented, and updated the patch08:30
ykarelas changing defaults there do not work08:30
amoralejbut08:30
amoralejare we setting it in every job?08:30
amoraleji see it now08:31
*** mosulica has joined #rdo08:31
ykarelyes nested_virt=false when fedora else true08:31
amoraleji'll remove it from weirdo08:31
amoralejand change default in weirdo-puppet08:31
amoralejmmm08:32
amoralejnot, that will not work08:32
ykarelhmm right, weirdo patch will fail on git_resource thing08:34
amoralejyes08:36
rdogerritAlfredo Moralejo proposed rdo-infra/ansible-role-weirdo-puppet-openstack master: Exclude git_resource from the list of puppet modules to install from rpm  https://review.rdoproject.org/r/1907708:36
amoralejykarel, ^ let's see that one08:36
ykarelamoralej, yup that should work08:37
*** Son_Goku has joined #rdo08:38
*** ksambor has quit IRC08:38
*** ksambor has joined #rdo08:40
*** xek has joined #rdo08:49
*** mosulica has quit IRC08:54
*** jpena|off is now known as jpena08:55
*** dpawlik has joined #rdo08:57
*** mosulica has joined #rdo08:58
*** jpich has joined #rdo08:58
*** Son_Goku has quit IRC08:59
*** dsneddon has joined #rdo08:59
*** gfidente has joined #rdo09:00
*** King_InuYasha has joined #rdo09:00
*** satya4ever has joined #rdo09:01
rdogerritFélix Enrique Llorente Pastora created config master: Force WORKSPACE same as we do in pre.yaml  https://review.rdoproject.org/r/1912309:02
rdogerritMerged config master: Force WORKSPACE same as we do in pre.yaml  https://review.rdoproject.org/r/1912309:06
*** ttsiouts has joined #rdo09:08
rdogerritMerged rdo-jobs master: Adds linting job  https://review.rdoproject.org/r/1769109:09
rdogerritYatin Karel proposed rdo-jobs master: [DNM] Rerun standalone job to test config change  https://review.rdoproject.org/r/1901709:12
*** ttsiouts has quit IRC09:13
*** numans has joined #rdo09:13
*** dpawlik has quit IRC09:17
*** wolverineav has quit IRC09:19
*** sshnaidm|afk has quit IRC09:26
ykarelamoralej, jpena let's get https://review.rdoproject.org/r/#/c/19077/09:28
*** chem has quit IRC09:30
rdogerritFélix Enrique Llorente Pastora created config master: Add dlrnapi_user var to post playbook  https://review.rdoproject.org/r/1912509:31
rdogerritFélix Enrique Llorente Pastora proposed config master: Add dlrnapi_user var to post playbook  https://review.rdoproject.org/r/1912509:31
*** mosulica has quit IRC09:34
rdogerritJavier Peña created rdo-jobs master: Remove bash shebang in weirdo-ansible-lint job  https://review.rdoproject.org/r/1912609:35
*** ykarel is now known as ykarel|lunch09:36
*** derekh has joined #rdo09:37
rdogerritJavier Peña proposed rdo-infra/ansible-role-weirdo-puppet-openstack master: Exclude git_resource from the list of puppet modules to install from rpm  https://review.rdoproject.org/r/1907709:41
*** mosulica has joined #rdo09:42
rdogerritJavier Peña created rdo-jobs master: Revert "Adds linting job"  https://review.rdoproject.org/r/1912709:45
rdogerritJavier Peña proposed rdo-infra/ansible-role-weirdo-puppet-openstack master: Exclude git_resource from the list of puppet modules to install from rpm  https://review.rdoproject.org/r/1907709:46
*** satya4ever is now known as sbulage|brb09:48
rdogerritMerged rdo-jobs master: Revert "Adds linting job"  https://review.rdoproject.org/r/1912709:53
rdogerritYatin Karel created rdoinfo master: Promote some deps to stein-testing  https://review.rdoproject.org/r/1912809:55
*** mosulica has quit IRC09:58
*** King_InuYasha has quit IRC10:00
*** holser_ has joined #rdo10:01
rdogerritMerged config master: Add dlrnapi_user var to post playbook  https://review.rdoproject.org/r/1912510:05
*** Son_Goku has joined #rdo10:07
*** holser_ has quit IRC10:10
*** holser_ has joined #rdo10:15
*** Son_Goku has quit IRC10:19
rdogerritFélix Enrique Llorente Pastora created config master: Use ansible-lint at tripleo playbooks/roles  https://review.rdoproject.org/r/1912910:22
rdogerritFélix Enrique Llorente Pastora proposed config master: Use ansible-lint at tripleo playbooks/roles  https://review.rdoproject.org/r/1912910:22
*** ykarel|lunch has quit IRC10:27
*** ykarel|lunch has joined #rdo10:28
*** mosulica has joined #rdo10:30
rdogerritMerged rdo-infra/ansible-role-weirdo-puppet-openstack master: Exclude git_resource from the list of puppet modules to install from rpm  https://review.rdoproject.org/r/1907710:38
*** zenirc369 has joined #rdo10:40
*** ramishra has quit IRC10:40
amoralejykarel|lunch, jpena ^ merged10:40
amoralejweirdo jobs should start passing now10:40
rdogerritAlfredo Moralejo proposed rdoinfo master: CBS tags sync with uc for stein-candidate (django)  https://review.rdoproject.org/r/1769810:44
ykarel|lunchamoralej, cool10:45
*** ykarel|lunch is now known as ykarel10:45
ykareldjango,10:45
*** sbulage|brb is now known as sbulage10:47
rdogerritAlfredo Moralejo created rdo-infra/ansible-role-weirdo-packstack master: Disable nested virtualization in weirdo-packstack until fixed  https://review.rdoproject.org/r/1913010:49
*** wolverineav has joined #rdo10:53
*** sshnaidm|afk has joined #rdo10:54
*** udesale has quit IRC10:57
*** sshnaidm|afk has quit IRC10:57
*** wolverineav has quit IRC10:57
*** xek has quit IRC10:58
*** xek_ has joined #rdo10:58
*** ramishra has joined #rdo10:59
rdogerritMerged openstack/sushy-distgit rpm-master: Requirement sync for Stein  https://review.rdoproject.org/r/1910211:00
*** pcaruana has quit IRC11:04
sfbenderJavier Peña proposed DLRN master: Use a regular expresion to avoid fallback to master  https://softwarefactory-project.io/r/1512111:07
*** leanderthal has joined #rdo11:07
sfbenderJavier Peña proposed DLRN master: Use a regexpn to avoid fallback to master  https://softwarefactory-project.io/r/1512111:07
sfbenderJavier Peña proposed DLRN master: Use a regexp to avoid fallback to master  https://softwarefactory-project.io/r/1512111:08
*** panda|ruck|off is now known as panda|ruck|flu11:10
rdogerritJavier Peña proposed rdo-infra/puppet-dlrn master: Add support for build_driver and some driver parameters  https://review.rdoproject.org/r/1431111:14
*** Son_Goku has joined #rdo11:14
*** morazi has quit IRC11:17
*** dciabrin_ has quit IRC11:19
*** dciabrin_ has joined #rdo11:19
panda|ruck|fluamoralej: periodic jobs yesterday were failing again with https://bugs.launchpad.net/tripleo/+bug/1818538 even after the bump, now they are working, did you have to do something additional ?11:24
openstackLaunchpad bug 1818538 in tripleo "All OVB jobs in master periodic pipeline fail to install the undercloud while trying to create the ctlplane network with 503 service unavailable on neutron" [Critical,Fix committed] - Assigned to Gabriele Cerami (gcerami)11:24
amoralejjpena, ykarel keep an eye on https://review.rdoproject.org/r/#/c/19130/ jobs are passing11:28
ykarelamoralej, ack11:29
amoralejpanda|ruck|flu, neutron runs in containers11:30
amoralejmaybe it's not updating undercloud containers properly?11:30
amoralejcan you check the version of neutron-lib in the containers?11:31
*** chem has joined #rdo11:31
*** pcaruana has joined #rdo11:32
ykareliirc old container-build job will fail as it would be using old tripleo-ci-testing tagged containers11:32
ykarelbecause it first installs undercloud then update containers11:32
panda|ruck|fluamoralej: mmhh, I'll check that, but now the jobs are passing the undercloud installation, so the patch worked11:34
amoralejgood11:34
*** jbrooks has quit IRC11:34
*** dsneddon has quit IRC11:35
*** Son_Goku has quit IRC11:41
ykarelamoralej, two job failed in packstack nested virt patch11:41
amoralejgrrr11:42
amoralejlet me check what happened11:42
ykarelboth fail with: Server 1771f9ac-b85c-488e-b3b1-218351eb89aa failed to reach ACTIVE status and task state "None" within the required time (300 s). Current status: BUILD. Current task state: spawning.11:42
*** Son_Goku has joined #rdo11:45
amoralejit may be just perf issue?11:49
amoraleji don't see any error in the logs11:52
quiquellamoralej, ykarel: Do you know if something has change at RDO hypervisors ? libguestfs-test is always failing now11:54
*** xek_ has quit IRC11:54
quiquellamoralej: for centos7 and fedora28 nested virtualization11:54
amoralejquiquell, yes, there is some problem with nested11:54
ykarelamoralej, hmm looks like perf, /me too can't find11:54
amoralejwe are trying to disable it11:54
quiquellamoralej: disabling it ?11:54
amoralejquiquell, yes11:54
quiquellamoralej: http://logs.rdoproject.org/47/19047/3/check/tripleo-ci-reproducer-fedora-28-libvirt/9768928/tripleo-ci-reproducer/libguestfs-test.log11:54
amoralejin weirdo jobs11:54
amoralejhttps://review.rdoproject.org/r/#/c/19130/11:54
amoralejhttps://review.rdoproject.org/r/#/c/19077/11:55
ykarelamoralej, as same test passing in other jobs, good to recheck and see11:55
amoralejkforde is working on it11:55
quiquellamoralej: will talk with kforde thanks!11:55
ykarelquiquell, you can see his mail11:55
ykarelhe sent some time back11:56
quiquellykarel: what mailing list ?11:56
ykarelquiquell, just checked it's personal mail, not to mailing list11:56
*** sshnaidm|afk has joined #rdo12:01
*** holser_ is now known as holser|lunch12:02
rdogerritFélix Enrique Llorente Pastora proposed rdo-infra/ansible-role-tripleo-ci-reproducer master: Add git review as dependency for reproducer  https://review.rdoproject.org/r/1904712:02
*** moguimar has quit IRC12:05
*** dsneddon has joined #rdo12:06
*** moguimar has joined #rdo12:10
*** dsneddon has quit IRC12:11
*** mvkr has joined #rdo12:13
rdogerritJavier Peña proposed rdo-infra/puppet-dlrn master: Add support for build_driver and some driver parameters  https://review.rdoproject.org/r/1431112:13
*** ttsiouts has joined #rdo12:16
quiquelljpena: o/ we have a local WARNING at ansible that does not appear at zuul12:30
quiquelljpena: maybe you can help us with that12:30
jpenaquiquell: sure, but it'll have to be after lunch :)12:30
jpenaI'm about to run12:31
quiquelljpena: of course no problem just ping me when ready12:31
*** jpena is now known as jpena|lunch12:31
*** kgiusti has joined #rdo12:35
*** udesale has joined #rdo12:38
*** kgiusti has quit IRC12:38
*** dsneddon has joined #rdo12:40
*** imcleod has joined #rdo12:41
weshay|roverianw any idea what may be causing https://bugs.launchpad.net/tripleo/+bug/181830512:43
openstackLaunchpad bug 1818305 in tripleo "overcloud-full image fails to build calling mkfs -t xfs, exec sudo failed" [Critical,Triaged] - Assigned to Gabriele Cerami (gcerami)12:43
*** dsneddon has quit IRC12:44
*** imcleod has quit IRC12:46
*** morazi has joined #rdo12:51
*** imcleod has joined #rdo12:52
*** holser|lunch is now known as holser_13:04
ykarelamoralej, so same jobs failed in packstack review13:05
*** TheJulia_sick is now known as TheJulia13:05
amoralejyeah13:05
amoraleji'm checking what happens13:05
ykarelsame tests, so it's seems some real issue13:05
*** dsneddon has joined #rdo13:05
ykarelack, /me looking too13:05
*** imcleod has quit IRC13:06
ykarelamoralej, atleast for ocata looks like inject_partition is related13:10
amoralejmmm13:10
amoralejthat rings me a bell13:10
ykareli remember in packstack we pushed it for some branches13:10
amoralejwhere do you see it?13:10
amoralejany error?13:10
ykarelnope didn't see any error, seeing pike and queens passing, ocata failing13:11
ykarelmade me think about inject_partition13:11
*** dsneddon has quit IRC13:11
*** mcornea has joined #rdo13:15
*** imcleod has joined #rdo13:17
*** ttsiouts has quit IRC13:17
*** ttsiouts has joined #rdo13:18
amoralejmaster is also failing, but logs looks different13:18
amoralejalthough the issus is also that it can't start instance13:18
*** imcleod has quit IRC13:21
*** rlandy has joined #rdo13:21
ykarelamoralej, hmm so it's stuck at Creating image13:23
*** ttsiouts has quit IRC13:23
ykarelamoralej, remember issue few days back?13:23
ykarelbackground: https://review.openstack.org/#/c/637684/13:25
ykarelwe faced same issue(2 test failed, last call was Creating image) in first run, recheck and local run passed13:26
*** imcleod has joined #rdo13:26
ykarellooking master logs now13:26
*** amoralej is now known as amoralej|lunch13:26
ykarelmmm in master it's different13:28
ykarelposting recheck, and continue looking logs13:28
*** ttsiouts has joined #rdo13:28
*** sshnaidm|afk is now known as sshnaidm13:32
*** jpena|lunch is now known as jpena13:34
jpenaquiquell: I'm back, how can I help?13:34
*** dsneddon has joined #rdo13:36
quiquelljpena: o/ so we had an issue at the periodic pipeline13:37
*** mjturek has joined #rdo13:37
quiquelljpena: so If I run this https://review.rdoproject.org/r/#/c/19124/13:38
quiquelljpena: note the missing dlrnapi_user variable13:38
ykarelamoralej|lunch, so master and ocata has different logs, but inject_partion=-1 in both case, pike/queens has -2, so seems that might cause the issue we are seeing13:38
quiquelljpena: zuul says nothing13:38
ykarelmay be jpena you remember the inject partition issue on ovh ^^13:38
jpenaykarel: not much, I remember we changed that for a reason, but that's it13:39
ykareljpena, ack :( i see the bug pointed there libguestfs, but don't have much info13:39
*** mjturek has quit IRC13:40
jpenaquiquell: have you tried printing the output of "env" in your test?13:41
jpenaI'm not sure if the ansible behavior in this case is to fail or just assume an empty var13:41
*** dsneddon has quit IRC13:42
jpenaquiquell: you mentioned a difference between openstack.org's zuul and RDO's, correct?13:42
quiquelljpena: running locally I get13:42
quiquellTASK [shell] *********************************************************************************************************13:42
quiquell [WARNING]: could not parse environment value, skipping: [u"{{ zuul | zuul_legacy_vars | combine({'DLRNAPI_PASSWORD':13:42
quiquelldlrnapi.password,'DLRNAPI_USERNAME': dlrnapi_user}) }}"]13:42
quiquellso don't know why zuul is not showing the same13:42
jpenawhich ansible version?13:42
*** quiquell is now known as quiquell|lunch13:42
quiquell|lunch2.7.513:43
jpenasee we have 2.5.14 in zuul: https://logs.rdoproject.org/24/19124/5/check/test-zuul-legacy-vars/42c2371/job-output.txt.gz#_2019-03-06_09_40_01_20366213:43
*** gfidente has quit IRC13:44
quiquell|lunchDamn it have to be it13:45
quiquell|lunchAck thanks13:45
quiquell|lunchWill check with that version13:46
*** dsneddon has joined #rdo13:46
*** pcaruana has quit IRC13:51
*** eharney has joined #rdo13:51
*** dsneddon has quit IRC13:54
*** sdake has quit IRC13:54
*** quiquell|lunch is now known as quiquell13:55
*** sdake has joined #rdo14:00
*** pcaruana has joined #rdo14:01
*** rfolco|pto is now known as rfolco|ruck14:02
ykareljpena, can you check if we have some instance in error state in rdo ci tenant14:03
jpenachecking14:03
rdogerritEmilien Macchi created openstack/tripleo-heat-templates-distgit rpm-master: Updates on directories  https://review.rdoproject.org/r/1913214:03
jpenaykarel: yes, 3 of them14:04
jpenaykarel: I can remove them14:04
ykareljpena, ack please clean14:05
ykareljpena, i saw ^^ in Delete task of weirdo14:05
ykarelDelete VMs in ERROR state or too old14:06
jpenaykarel: deleted14:07
jpenathere's a VM running for more than a day, I think we'll have to remove it too14:07
ykarelis it running for a job?14:08
*** wolverineav has joined #rdo14:08
*** Son_Goku has quit IRC14:11
ykarelso those were not cleaned because of same name conflict and delete is using item.name14:12
*** wolverineav has quit IRC14:12
ykarelMultiple matches found for generic-puppet-openstack-004-495414:12
ykareljpena, any reason to not use .id for deletion14:13
jpenaykarel: no, I don't remember why we did it like that14:13
ykarelhttps://github.com/rdo-infra/ci-config/blob/63214f36f97463d0692b3200dd5ec1997c2a6626/jenkins/jobs/scripts/create-vm.sh#L8114:13
ykarelok, not urgent can check that later, currently we have other bigger issues14:14
*** baha has joined #rdo14:16
*** Son_Goku has joined #rdo14:16
*** gkadam_ has joined #rdo14:16
*** sdake has quit IRC14:17
*** gkadam has quit IRC14:19
*** rascasoft has quit IRC14:20
rdogerritJon Schlueter created openstack/oslo-concurrency-distgit rpm-master: remove Req on python3-fixtures from main package  https://review.rdoproject.org/r/1913314:21
*** sdake has joined #rdo14:23
*** dsneddon has joined #rdo14:24
*** sdake has quit IRC14:24
*** openstackgerrit has joined #rdo14:26
*** ChanServ sets mode: +v openstackgerrit14:26
openstackgerrityatin proposed openstack/packstack master: Switch to default -2 for libvirt_inject_partition  https://review.openstack.org/64138714:26
ykareljpena, amoralej|lunch proposed ^^14:26
*** jcoufal has joined #rdo14:27
*** amoralej|lunch is now known as amoralej14:29
*** dsneddon has quit IRC14:29
rdogerritJohn Eckersberg proposed rdoinfo master: Add python-etcd3gw to stein-testing  https://review.rdoproject.org/r/1905014:30
*** mjturek has joined #rdo14:30
jpenathe RDO meeting will start in ~30 minutes. Remember to add topics to the agenda at https://etherpad.openstack.org/p/RDO-Meeting14:32
rdogerritFélix Enrique Llorente Pastora proposed rdo-infra/ansible-role-tripleo-ci-reproducer master: Add git review as dependency for reproducer  https://review.rdoproject.org/r/1904714:32
rdogerritSagi Shnaidman created rdo-infra/ansible-role-tripleo-ci-reproducer master: DNM: try job on vexxhost  https://review.rdoproject.org/r/1913414:34
sshnaidmquiquell, ^^14:34
rdogerritAlfredo Moralejo proposed rdoinfo master: Bump rdoinfo stein-uc tags to latest upper-constraints  https://review.rdoproject.org/r/1896514:35
quiquellsshnaidm: let's see14:36
*** sdake has joined #rdo14:38
*** kgiusti has joined #rdo14:39
rdogerritJon Schlueter created openstack/oslo-i18n-distgit rpm-master: remove Req on python[23]-fixtures from main package  https://review.rdoproject.org/r/1913514:40
openstackgerrityatin proposed openstack/packstack master: [DNM] Debug libguestfs issue  https://review.openstack.org/64139714:40
ykarelamoralej, also attempting ^^ to see if we can get more info14:40
*** dsneddon has joined #rdo14:42
*** rascasoft has joined #rdo14:43
rdogerritJon Schlueter created openstack/ovsdbapp-distgit rpm-master: Move req python[*]-fixtures to -test subrpm  https://review.rdoproject.org/r/1913614:46
*** gfidente has joined #rdo14:47
*** dsneddon has quit IRC14:47
jschlueterykarel, amoralej: I have a couple more python-fixtures Requires on main rpm patches proposed, would be nice to get eyes on them and landed soon14:49
ykareljschlueter, sure14:49
jschlueterhttps://review.rdoproject.org/r/#/q/topic:fixtures_on_rpm+(status:open+OR+status:merged)14:49
amoralejyeah, i saw them14:49
jschlueteramoralej: thanks guys14:49
jschlueterpull in others if we need to for review just I know these are trivial changes14:49
*** rascasoft has quit IRC14:50
*** rascasoft has joined #rdo14:51
jschlueteror should be trivial changes14:51
*** ktibi has joined #rdo14:53
rdogerritJavier Peña created rdo-infra/puppet-dlrn master: stable/stein bootstrap is over  https://review.rdoproject.org/r/1913714:53
*** dprince has joined #rdo14:53
*** gkadam__ has joined #rdo15:00
*** sdake has quit IRC15:00
jpenait's meeting time15:00
ykarelso now we have NODE_FAILURE15:00
jpena#startmeeting RDO meeting - 2019-03-0615:00
openstackMeeting started Wed Mar  6 15:00:56 2019 UTC and is due to finish in 60 minutes.  The chair is jpena. Information about MeetBot at http://wiki.debian.org/MeetBot.15:00
openstackUseful Commands: #action #agreed #help #info #idea #link #topic #startvote.15:00
*** openstack changes topic to " (Meeting topic: RDO meeting - 2019-03-06)"15:00
openstackThe meeting name has been set to 'rdo_meeting___2019_03_06'15:01
jpena#topic roll call15:01
*** openstack changes topic to "roll call (Meeting topic: RDO meeting - 2019-03-06)"15:01
amoralejo/15:01
jpenaremember to add any last-minute topic to the agenda: https://etherpad.openstack.org/p/RDO-Meeting15:01
jpena#chair amoralej ykarel15:01
openstackCurrent chairs: amoralej jpena ykarel15:01
ykarelo/15:01
mjtureko/15:01
PagliaccisCloudo/15:01
jpena#chair mjturek PagliaccisCloud15:01
openstackCurrent chairs: PagliaccisCloud amoralej jpena mjturek ykarel15:01
bahao/15:01
jpena#chair baha15:02
openstackCurrent chairs: PagliaccisCloud amoralej baha jpena mjturek ykarel15:02
*** fultonj has joined #rdo15:02
*** chandankumar is now known as chkumar|pto15:03
*** gkadam_ has quit IRC15:03
jpenalet's start with the agenda15:04
jpena#topic Status of Stein release15:04
*** openstack changes topic to "Status of Stein release (Meeting topic: RDO meeting - 2019-03-06)"15:04
rdogerritAlfredo Moralejo created rdo-infra/ci-config master: Move weirdo jobs to ci aggretate temporarily  https://review.rdoproject.org/r/1913815:04
jpenaamoralej: ^^15:04
rdogerritMerged openstack/ovsdbapp-distgit rpm-master: Move req python[*]-fixtures to -test subrpm  https://review.rdoproject.org/r/1913615:04
amoralejit's just to share the status and plans for Stein GA preparation tasks15:05
amoralejjpena has already created new DLRN builders following stable/stein branches15:05
amoralejboth for centos7 and fedora-stabilized15:05
*** sdake has joined #rdo15:05
amoralejbootstrap is finished15:06
amoralejand both are consistent15:06
amoralejcurrently are available in http://trunk.rdoproject.org/centos7-stein-bootstrap15:06
amoralejand http://trunk.rdoproject.org/fedora28-stein-bootstrap15:06
amoralejtomorrow we'll switch centos7-stein and fedora28-stein links to the new builders15:07
amoralej#info DLRN builders for CentOS7 and Fedora28 chasing stable/stein branches are bootstrapped15:07
amoralej#info URLs http://trunk.rdoproject.org/centos7-stein and http://trunk.rdoproject.org/fedora28-stein will be switched to new builders tomorrow15:08
amoralejany user that wants to use master branches must use centos7-master15:08
amoralejapart from that, we are already starting requirements updates for stein15:08
amoralej#info requirements adjustements for stein ongoing in https://review.rdoproject.org/r/#/q/topic:stein-branching15:09
amoralejwe should start doint it for libraries15:09
amoralejand clients at the end of this week15:09
amoralejand we will start branching distgits and building libraris early next week15:10
amoralej#info we will start branching distgits and building libraris early next week15:10
amoralej#action maintainers send required adjustments for distgits for stein15:10
ykarelamoralej, do you know when stable/stein branches created upstream, i see only few created till now15:10
amoraleji think that's it15:11
amoralejykarel, for libraries and clients, it's not a big problem to have the branches created because they are pinned15:11
amoralejfor others, it should be in rc115:11
ykarelamoralej, yup, was unsure about the timing, so asked15:12
amoralejwe are also finishing deps rebases for stain15:12
amoralejyeah, it should be soon15:12
weshay|rover0/15:12
ykarelack15:12
amoraleji think that's it from me15:13
ykarel#link status for release at https://review.rdoproject.org/etherpad/p/stein-release-preparation15:13
*** dsneddon has joined #rdo15:13
jpenaok, let's move on then15:13
amoralejthanks ykarel15:13
jschluetertempest is one of few that still has dep on os-testr and not yet flipped to stestr?15:13
jpena#topic open floor15:13
*** openstack changes topic to "open floor (Meeting topic: RDO meeting - 2019-03-06)"15:13
weshay|roveryou guys are ready for CI to start building ci jobs for stein it seems, right?15:13
ykareltempest should be switched i think15:13
weshay|roverI see repos w/ links :)15:13
amoralejweshay|rover, almost15:13
amoralejyes15:14
weshay|roveramoralej++15:14
*** agopi has joined #rdo15:14
*** sbulage is now known as sbulage|afk15:14
weshay|roverone other qq15:14
amoralejwe are consistent, we just need to switch links15:14
*** sbulage|afk has quit IRC15:14
amoralejwe expect to do it tomorrow, i'll send a mail to the ML15:14
weshay|roverPike, does rdo need more builds of Pike due to the extended release?15:14
* Duck o/15:14
weshay|roverthere are issues w/ ceph that will require a bit of work to promote another build15:14
weshay|roveramoralej k.. thanks15:14
amoralejweshay|rover, you mean pike with ceph is broken?15:15
weshay|roveron unpromoted builds yes15:15
amoralejaccording to https://releases.openstack.org/ pike is still maintained15:15
weshay|roverso there is a significant cost to not EOL pike atm15:15
*** ekultails has joined #rdo15:16
amoralejso i'd like to get pike dlrn builder running15:16
ykarelpromoted builds should also be affected i think15:16
amoralejwhat introduced the problem? update in ceph?15:16
weshay|roverfultonj can you comment15:16
weshay|roveramoralej lp 181768815:17
openstackLaunchpad bug 1817688 in tripleo "job that test ceph in periodic pipeline for stable branches are getting the wrong configuration" [Critical,Fix released] https://launchpad.net/bugs/1817688 - Assigned to Gabriele Cerami (gcerami)15:17
fultonjyes15:17
fultonjtripleo ci can't fully switch on versions15:17
amoralejoh, that i remember now15:17
fultonjwe switched ceph from filestore to bluestore for all versions which support it15:18
fultonjpike doesn't because pike is aligned with jewel15:18
*** dsneddon has quit IRC15:18
amoralejthere is no easy wat to switch back in pike?15:18
rdogerritSagi Shnaidman proposed rdo-infra/ansible-role-tripleo-ci-reproducer master: Run centos jobs on vexxhost  https://review.rdoproject.org/r/1913415:18
*** agopi has quit IRC15:18
fultonjeasy no15:18
amoralejweshay|rover, so you are proposing to stop promoting pike, iiuc15:19
*** jbrooks has joined #rdo15:19
fultonjtripleo ci can get an option to switch on version at a cost15:19
fultonjis value(cost) > keeping pike15:19
fultonjthat's the question?15:19
weshay|roveramoralej pretty much yes15:20
amoralejwell, it's almost moving to extended maintenance15:20
amoraleji'd keep dlrn builders running15:21
amoralejso that user can consume package updates15:21
*** quiquell is now known as quiquell|off15:21
amoralejeven if it's for non-ceph or even non-tripleo environments15:21
fultonjoption: keep builder but stop CI'ing ceph jobs on it15:21
weshay|roverI suppose we could remove ooo from pike criteria15:22
fultonjweshay|rover: perhaps not all ooo but just ooo w/ ceph?15:22
amoralejor maybe ooo+ceph15:22
fultonjfor pike only15:22
amoralejwould that work?15:22
amoralejfor pike15:22
weshay|roverI think it's a viable option15:22
fultonjwho would remove the ooo+ceph from pike?15:23
fultonjis that a weshay|rover thing or an amoralej thing?15:24
weshay|roverfultonj we should look at that together15:24
ykarelfultonj, so iiuc tasks in https://review.openstack.org/#/c/637196/6/roles/ceph-loop-device/tasks/main.yml caused it15:24
ykareland specifically using ceph_loop_device15:25
fultonjykarel: yeah i know15:25
fultonjsee line 40 it doesnt' work15:25
ykarelso using different loop_device will not solve it15:25
ykarelyup i know that, release is master in periodic jobs15:25
amoraleji think we could update periodic pipeline and promoter to just remove ceph jobs15:26
fultonjamoralej: ++15:26
fultonjdifferent loop_device ?15:26
fultonjthat's horrible and ugly but it might work15:26
ykarelhmm that what i got from error it's used somewhere15:27
ykarelso using different might be a workaround15:27
ykarelbut ok to remove jobs and keep dlrn running15:27
amoralej+1 to remove the job, even removing the full promotion is not a big problem imo15:28
fultonjykarel++ good idea15:29
fultonji'll try the workaround15:29
fultonjnext step me15:29
fultonji'll update this group with results15:29
fultonjif it doesnt' work let's go to plan b15:29
fultonj remove jobs and keep dlrn running15:29
fultonjok?15:29
fultonjweshay|rover: amoralej ykarel ?15:30
ykarel+1 for me15:30
*** xek_ has joined #rdo15:30
gfidente+1 fwiw15:30
weshay|rovercool, fultonj I'll sync w/ u15:30
jpenaany action to record?15:31
fultonj#ACTION fultonj to try second loop devices for https://review.openstack.org/#/c/637196/ and pike and report if it fixes pike job xor we remove ooo+ceph tests for pike only15:31
fultonji'd like to talk about https://review.rdoproject.org/r/#/c/18721/ at the approipriate time15:32
ykarelfultonj, you can test failing pike job like https://review.rdoproject.org/r/#/c/19017 with depends on on tripleo-ci patch just in case u are not aware15:32
fultonjas anothe topic15:32
jpena#agreed keep RDO Trunk pike but remove ceph from pike promotion criteria15:32
*** xek__ has joined #rdo15:32
jpenafultonj: the previous topic is done, feel free to start with ansible15:33
fultonj#TOPIC ansible 2.7 in rdo for train https://review.rdoproject.org/r/#/c/18721/15:33
jpena#topic  ansible 2.7 in rdo for train https://review.rdoproject.org/r/#/c/18721/15:34
*** openstack changes topic to "ansible 2.7 in rdo for train https://review.rdoproject.org/r/#/c/18721/ (Meeting topic: RDO meeting - 2019-03-06)"15:34
jpenait's lowercase :D15:34
fultonjthanks jpena15:34
amoralejany update on nautilus release?15:34
fultonjwe're blocked on needing newer clients15:34
fultonjnautilus clients15:34
gfidenteyeah I can't get a build to pass in centos/cbs15:34
gfidentehave to retry15:34
*** xek_ has quit IRC15:35
fultonjcan we get help w/ why 1. https://cbs.centos.org/koji/getfile?taskID=710313&name=build.log&offset=-4000 fails?15:35
fultonjktdreyer: ^ :)15:35
fultonjthat's the status update. https://review.rdoproject.org/r/#/c/18721 is blocked on getting clients on failure ^15:37
*** udesale has quit IRC15:37
fultonjso should we move on?15:41
jpenayes, let's continue15:41
jpena#topic chair for next meeting15:41
*** openstack changes topic to "chair for next meeting (Meeting topic: RDO meeting - 2019-03-06)"15:41
jpenado we have any volunteer to chair next week's meeting?15:42
mjturekjpena: I can run15:43
jpenathanks mjturek!15:43
jpena#action mjturek to chair the next meeting15:43
mjtureknp15:44
jpenaok, so unless we have any other topic to bring, let's end the meeting and put those extra 15 minutes to good use15:44
rdogerritMerged openstack/oslo-concurrency-distgit rpm-master: remove Req on python3-fixtures from main package  https://review.rdoproject.org/r/1913315:44
mjturekoh I do!15:44
mjturekbaha and I are looking for the proper way to get the IP of the jenkins slave running the job for ci.centos.org through ansible. We need it to properly rsync the logs to the logserver. Anyone have any insight here?15:44
jpenamjturek: do you need to get it from the cico node, or from the slave itself?15:45
mjturekthe slave itself!15:45
jpenathere must be an ansible fact you can use, let me check15:47
*** dsneddon has joined #rdo15:47
mjturekjpena: maybe this ansible_default_ipv4.address?15:48
jpenamjturek: yep, that's what I was thinking about15:49
mjturekcool!15:49
mjturekand I think it would just need to run on localhost to properly resolve right?15:49
jpenayes15:50
*** dtantsur|mtg is now known as dtantsur|afk15:50
mjturekcool! I think we're good then, thanks jpena15:50
jpenaso this is everyone's last chance of raising a topic!15:50
rdogerritMerged openstack/tripleo-heat-templates-distgit rpm-master: Updates on directories  https://review.rdoproject.org/r/1913215:51
*** pcaruana has quit IRC15:53
jpena3, 2, 1...15:53
jpenagone!15:53
jpena#endmeeting15:53
*** openstack changes topic to "RDO: An OpenStack Distribution for CentOS/RHEL | Community meeting is held here every Wednesday at 15:00UTC | Please read our IRC community guidelines: https://www.rdoproject.org/community/irc-etiquette/ | Logged at http://eavesdrop.openstack.org/irclogs/%23rdo/"15:53
openstackMeeting ended Wed Mar  6 15:53:13 2019 UTC.  Information about MeetBot at http://wiki.debian.org/MeetBot . (v 0.1.4)15:53
openstackMinutes:        http://eavesdrop.openstack.org/meetings/rdo_meeting___2019_03_06/2019/rdo_meeting___2019_03_06.2019-03-06-15.00.html15:53
openstackMinutes (text): http://eavesdrop.openstack.org/meetings/rdo_meeting___2019_03_06/2019/rdo_meeting___2019_03_06.2019-03-06-15.00.txt15:53
openstackLog:            http://eavesdrop.openstack.org/meetings/rdo_meeting___2019_03_06/2019/rdo_meeting___2019_03_06.2019-03-06-15.00.log.html15:53
jpenathanks for attending15:53
amoralejthanks jpena15:54
*** janki has quit IRC15:55
*** Son_Goku has quit IRC15:56
*** Son_Goku has joined #rdo15:58
*** mosulica has quit IRC15:58
*** Son_Goku has quit IRC16:01
ykarelthanks jpena16:05
*** pcaruana has joined #rdo16:06
*** ttsiouts has quit IRC16:08
*** ttsiouts has joined #rdo16:09
*** sdake has quit IRC16:09
*** ttsiouts has quit IRC16:11
*** ttsiouts has joined #rdo16:11
*** dciabrin_ has quit IRC16:12
*** dciabrin__ has joined #rdo16:12
*** Vorrtex has joined #rdo16:13
*** sdake has joined #rdo16:14
*** amuller has joined #rdo16:23
zbrykarel: i seen a msg related to linting from quiquell|off - do you know which repo he was referring about? i would glad to help on that.16:25
ykarelzbr, not sure what quiquell|off shared you, there were two repos being discussed today: config and rdo-jobs16:26
*** ccamacho has quit IRC16:27
zbrykarel: do we have already open changes on that or not? either way I can help to quickly adop it. funny, I had the same two repos in mind myself.16:27
ykarelrdo-jobs linting was reverted today as some job failed post it, and in config linting is not there yet, and to catch issues like https://review.rdoproject.org/r/#/c/19121/ quiquell|off suggested to have linting16:28
*** ksambor has quit IRC16:31
zbrykarel: yeah. that is why I usually introduce it in batches, to avoid accidents caused by bugs with too big reviews.16:31
zbrykarel: simple https://review.rdoproject.org/r/#/c/18631/16:31
ykarelit's different, we were talking about config, not ci-config16:32
gfidentedo you guys know16:32
gfidenteif I can download the srpm16:32
gfidenteof a build in centos/cvs16:32
gfidentecentos/cbs16:32
gfidentehttps://cbs.centos.org/koji/buildinfo?buildID=2514116:32
gfidente?16:32
ykarelatleast when build is successful, srpm exists16:33
gfidente Keep SRPM? no16:34
gfidente:)16:34
gfidentesigh16:34
ykarelsrpm can be created from spec though16:34
*** leanderthal has quit IRC16:34
gfidenteykarel it's the .spec I wanted! :16:34
ykarelohhk16:35
amoralejjpena, ykarel could you take a look into https://review.rdoproject.org/r/#/c/19138/ ?16:35
*** imacdonn has quit IRC16:36
openstackgerritMerged openstack/packstack master: Switch to default -2 for libvirt_inject_partition  https://review.openstack.org/64138716:36
*** imacdonn has joined #rdo16:36
rdogerritYatin Karel proposed rdoinfo master: Promote some deps to stein-testing  https://review.rdoproject.org/r/1912816:36
ykarelamoralej, kforde just pinged on rhos-ops he rebooted rdo aggregate compute16:37
ykarelamoralej, running jobs with ^^16:37
amoralejok16:37
ykarelamoralej, if job still fails, let's switch to ci.m1 flavor16:37
ykareland report to kforde16:37
amoralejlet me try to create an instance manually16:38
ykarelokk16:38
zbrykarel: sshnaidm : quique did the right thing on https://review.rdoproject.org/r/#/c/19129/ - lets merge it and I can build on top of it.16:39
*** gkadam__ has quit IRC16:42
*** jfrancoa has quit IRC16:46
amoralejykarel, yeah it seems to be working fine now16:47
ykarelamoralej, hmm looking16:47
ykarelvms created?16:47
amoraleji created an instance16:47
amoralejfrom horizon16:47
amoralejit went fine16:47
amoralejand it was failing before16:47
ykarelokk16:47
*** sdake has quit IRC16:48
*** sdake has joined #rdo16:48
*** pcaruana has quit IRC16:55
*** aufi has quit IRC16:55
*** sdake has quit IRC16:56
*** zenirc369 has quit IRC16:56
*** panda|ruck|flu is now known as panda|ruck|off16:57
*** paramite has quit IRC16:58
*** ccamacho has joined #rdo17:02
*** eharney has quit IRC17:06
*** eharney has joined #rdo17:13
*** ramishra has quit IRC17:14
*** ccamacho has quit IRC17:15
*** ccamacho has joined #rdo17:19
*** ykarel is now known as ykarel|away17:20
*** rlandy is now known as rlandy|brb17:21
*** ttsiouts has quit IRC17:24
*** ttsiouts has joined #rdo17:25
*** hseipp has quit IRC17:27
*** ttsiouts has quit IRC17:29
*** dsneddon has quit IRC17:33
*** kopecmartin is now known as kopecmartin|off17:33
*** jpich has quit IRC17:37
*** rlandy|brb is now known as rlandy17:38
*** baha has quit IRC17:42
*** wolverineav has joined #rdo17:44
*** wolverineav has quit IRC17:49
*** fkluknav has quit IRC17:49
rdogerritwes hayutin created config master: add ovb to test image build  https://review.rdoproject.org/r/1913917:51
*** fkluknav has joined #rdo17:52
*** wolverineav has joined #rdo17:54
*** ccamacho has quit IRC18:00
*** ktibi has quit IRC18:01
*** baha has joined #rdo18:01
*** wolverineav has quit IRC18:01
*** Son_Goku has joined #rdo18:02
*** dsneddon has joined #rdo18:03
*** derekh has quit IRC18:03
*** wolverineav has joined #rdo18:06
*** jtomasek has quit IRC18:06
*** wolverineav has joined #rdo18:06
*** amuller has quit IRC18:13
*** dciabrin__ is now known as dciabrin18:16
*** jpena is now known as jpena|off18:19
rdogerritMerged openstack/oslo-i18n-distgit rpm-master: remove Req on python[23]-fixtures from main package  https://review.rdoproject.org/r/1913518:25
rdogerritwes hayutin proposed config master: add ovb to test image build  https://review.rdoproject.org/r/1913918:25
*** amoralej is now known as amoralej|off18:26
*** moguimar has quit IRC18:33
*** ccamposr has quit IRC18:35
*** holser_ has quit IRC18:45
*** mburned is now known as mburned_out18:50
*** mburned_out is now known as mburned18:51
*** baha has quit IRC18:51
*** mjturek has quit IRC18:51
rdogerritwes hayutin proposed config master: add ovb to test image build  https://review.rdoproject.org/r/1913918:59
*** aedc has joined #rdo19:03
*** morazi has quit IRC19:09
*** baha has joined #rdo19:12
*** mjturek has joined #rdo19:13
*** wolverineav has quit IRC19:14
*** zaneb has quit IRC19:15
*** ykarel_ has joined #rdo19:20
*** ykarel|away has quit IRC19:22
*** wolverineav has joined #rdo19:25
*** rfolco|ruck has quit IRC19:26
*** wolverineav has quit IRC19:26
*** gfidente has quit IRC19:26
*** wolverineav has joined #rdo19:26
*** rfolco has joined #rdo19:27
*** wolverineav has quit IRC19:29
*** gfidente has joined #rdo19:30
*** wolverineav has joined #rdo19:30
*** baha has quit IRC19:34
*** baha has joined #rdo19:34
*** wolverineav has quit IRC19:35
*** wolverineav has joined #rdo19:41
*** xek has joined #rdo19:42
rdogerritMerged config master: add ovb to test image build  https://review.rdoproject.org/r/1913919:43
*** xek has quit IRC19:44
*** xek_ has joined #rdo19:44
*** xek__ has quit IRC19:45
*** wolverineav has quit IRC19:47
sfbenderJon Schlueter proposed rdopkg master: Refactor to create search_bug_references function  https://softwarefactory-project.io/r/972519:52
*** wolverineav has joined #rdo19:56
*** tosky has quit IRC20:00
jruzickaoooh, contributions! :D20:00
*** tosky has joined #rdo20:01
*** mjturek has quit IRC20:06
*** mjturek has joined #rdo20:07
rdogerritMerged rdoinfo master: Bump rdoinfo stein-uc tags to latest upper-constraints  https://review.rdoproject.org/r/1896520:10
rdogerritMerged rdoinfo master: CBS tags sync with uc for stein-candidate (django)  https://review.rdoproject.org/r/1769820:10
*** mjturek has quit IRC20:11
*** mjturek has joined #rdo20:12
*** dprince has quit IRC20:15
*** wolverineav has quit IRC20:21
fultonjykarel_: https://review.openstack.org/#/c/641479/20:29
*** wolverineav has joined #rdo20:37
*** jlibosva has quit IRC20:39
rdogerritJohn Fulton proposed rdo-jobs master: [DNM] Rerun standalone job to test config change  https://review.rdoproject.org/r/1901720:40
*** Son_Goku has quit IRC20:47
*** zbr|ssbarnea has joined #rdo21:07
*** zbr has quit IRC21:10
rdogerritAdam Kimball created rdo-infra/ci-config master: Add logging to ppc64le Triple O containers job  https://review.rdoproject.org/r/1914021:14
*** zaneb has joined #rdo21:15
rdogerritAdam Kimball proposed rdo-infra/ci-config master: Add logging to ppc64le Triple O containers job  https://review.rdoproject.org/r/1914021:15
*** mjturek has quit IRC21:16
*** baha has quit IRC21:18
*** eharney has quit IRC21:29
*** Vorrtex has quit IRC21:39
*** mjturek has joined #rdo21:39
*** baha has joined #rdo21:40
ianwweshay|rover : looking, i'm fairly sure we haven't changed anything too related lately21:40
weshay|roverianw aye.. having trouble reproducing it on a local instance.. however it's all over our ci21:42
weshay|roverthis page takes 2min to load, but you can see in http://cistatus.tripleo.org/  ---> click on tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset00121:43
weshay|roverafter it finally loads,.. and search for "overcloud image" in the search box on the right21:43
weshay|roverI'm 2/2 successfully building locally21:43
ianw Create image file [/tmp/dib_image.zFsUnNfg/image0.raw]21:44
rdogerritUser mjturek proposed rdo-infra/ci-config master: Add logging to ppc64le Triple O containers job  https://review.rdoproject.org/r/1914021:44
ianwit's a pretty big image right?21:44
weshay|roverI can try to check21:44
weshay|rovernot sure21:44
ianwwondering if something has pushed it over what /tmp is happy with.  for infra etc we have the tmp/intermediate stuff on a temp dir on disk21:45
weshay|roveron my system that work it's just a temp file and gone21:45
ianwyeah, i mean it *should* be fine ... clutching at straws :)21:46
weshay|roverI can try to spin up an env in rdo and check the /tmp size21:46
weshay|roverya.. same here21:46
ianwit seems there's no partition being created, is that correct?21:49
weshay|roverianw not sure.. we turned on debug here https://review.openstack.org/#/c/641417/ but the build also did not fail there21:51
weshay|roverwe should know more once this merges21:51
weshay|roveroh.. u found that21:51
* weshay|rover reads21:51
weshay|roverianw do you know which element is running here?21:53
weshay|roverINFO diskimage_builder.block_device.level0.localloop [-] Remove image file [/tmp/dib_image.nToRxFIU/image0.raw]21:53
weshay|rover2019-03-06 12:43:15 | 2019-03-06 12:43:15.102 | ERROR diskimage_builder.block_device.blockdevice [-] Rollback complete, exiting21:53
ianwthat's not an element as such, that's the block-device stuff that makes the final images21:53
weshay|roverhttp://git.openstack.org/cgit/openstack/diskimage-builder/tree/diskimage_builder/block_device/level0/localloop.py#n3621:53
ianwwow that page is a browser killer :)21:54
ianwyeah, that's a pretty well trodden path creating the final image, partitions and file systems, then it gets wrapped into the final image21:56
weshay|roveragree, and a pretty simple call21:58
weshay|roverianw oh wait.. so if the create failed21:58
weshay|roverthat's right..21:59
weshay|roverit's this that is failing21:59
weshay|roversudo mkfs -t xfs -s size=4096 -L img-rootfs -m uuid=2186f706-2942-413b-849c-5d9742e5b52e -q /dev/loop021:59
weshay|rovernot the delete21:59
weshay|roverso maybe it is the size21:59
*** jcoufal has quit IRC21:59
weshay|roversorry.. hadn't looked at this for a bit21:59
ianwthe last "success" i can find is https://logs.rdoproject.org/46/618546/5/openstack-check/tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset001/d1f62cf/, i think22:00
*** fmount has quit IRC22:03
*** fmount has joined #rdo22:04
*** apetrich has quit IRC22:08
rdogerritAlan Pevec proposed rdoinfo master: bump Ansible to 2.7 for Stein  https://review.rdoproject.org/r/1872122:08
*** apetrich has joined #rdo22:11
ianwso looking between working and not working, no major differences : http://paste.openstack.org/show/747378/22:11
ianwthe failing one was 2.20.2 ... the working 2.20.3 ; that was only a gentoo release key, so that's not related22:13
weshay|roverianw hrm.. k.. we're getting an env up to check /tmp22:13
ianwsome sort of hotplug type thing noticing the new loopback device and doing ... something?22:15
ianwit's still getting /dev/loop0 so it seems unlikely we've somehow got some sort of loopback device race going on with something else external22:16
weshay|roverthat could be22:21
weshay|roverseems to only happen in rdo-cloud22:21
weshay|roverianw mwhahaha I think we can safely say it's something to do w/ the env in which it's running..  so we'll go try ovb / rdo nodes now22:29
weshay|roverthanks for your help you guys22:29
rdogerritJohn Fulton created rdo-jobs master: [DNM] Rerun standalone job to test config change  https://review.rdoproject.org/r/1914122:38
rdogerritJohn Fulton proposed rdo-jobs master: [DNM] Rerun standalone job to test config change  https://review.rdoproject.org/r/1901722:40
ianwweshay|rover: yeah, logging, and if we can hold a node can probably figure out.  otherwise i don't have any more great ideas :)22:41
rdogerritJohn Fulton proposed rdo-jobs master: [DNM] Rerun standalone job to test config change  https://review.rdoproject.org/r/1901722:42
*** baha has quit IRC22:42
rdogerritAlan Pevec proposed rdoinfo master: bump Ansible to 2.7 for Stein  https://review.rdoproject.org/r/1872122:42
*** mcornea has quit IRC22:48
*** mjturek has quit IRC22:51
*** rcernin has joined #rdo22:52
*** wolverineav has quit IRC22:52
rdogerritSagi Shnaidman proposed rdo-infra/ansible-role-tripleo-ci-reproducer master: Run centos jobs on vexxhost  https://review.rdoproject.org/r/1913422:54
*** wolverineav has joined #rdo22:58
*** wolverineav has quit IRC23:13
*** wolverineav has joined #rdo23:17
*** aedc has quit IRC23:21
*** aedc has joined #rdo23:23
*** rlandy is now known as rlandy|bbl23:24
*** ekultails has quit IRC23:29
*** stee has joined #rdo23:30
*** Son_Goku has joined #rdo23:30
*** rascasoft has quit IRC23:34
*** Son_Goku has quit IRC23:34
*** rascasoft has joined #rdo23:37
*** rh-jelabarre has quit IRC23:43
*** Son_Goku has joined #rdo23:49

Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!