Tuesday, 2018-10-16

*** rlandy is now known as rlandy|bbl00:03
*** agopi has quit IRC00:29
*** agopi has joined #oooq00:30
*** ssbarnea has joined #oooq00:48
*** ssbarnea|bkp2 has quit IRC00:48
*** agopi_ has joined #oooq00:49
*** agopi has quit IRC00:50
*** rlandy|bbl is now known as rlandy01:28
*** jschlueter has quit IRC01:56
*** jschlueter has joined #oooq02:01
*** chem has quit IRC02:50
*** rlandy has quit IRC03:15
*** agopi_ is now known as agopi04:18
*** ykarel has joined #oooq04:22
*** ratailor has joined #oooq05:30
*** jfrancoa has joined #oooq05:38
*** apetrich has quit IRC05:40
*** apetrich has joined #oooq06:00
*** saneax has joined #oooq06:02
*** chkumar|off is now known as chandankumar06:05
*** apetrich has quit IRC06:23
*** apetrich has joined #oooq06:44
*** jtomasek has joined #oooq07:05
*** quiquell|off is now known as quiquell07:24
*** amoralej|off is now known as amoralej07:28
quiquellssbarnea: Good morning07:29
ssbarneaquiquell: morning07:39
quiquellssbarnea: So at the end the gather thing was the issue ?07:40
*** bogdando has joined #oooq07:40
ssbarneayep, i think so. the become on it07:40
ssbarneai would say it deserve an ansible bug07:41
quiquellssbarnea: So it does a sudo and continue with it ?07:41
quiquellssbarnea: That's super weird07:41
quiquellssbarnea: Going to rebase the changes to be able to run zuul07:46
quiquellssbarnea: is ok ?07:46
ssbarneaquiquell: yep i already started this07:47
quiquellssbarnea: Ok done let's see if now we don' thave a conflict07:47
ssbarneaquiquell marios : please review the linting fix: it passed zuul https://review.openstack.org/#/c/610501/07:48
ssbarneai want it merged before someone else is breaking it07:48
ssbarneaif we don't merge it fast, something else will broke it by introducing bad syntax07:48
marios|roverack ssbarnea already in my todos :)07:48
marios|roverssbarnea: the skip only merged like this morning !07:48
marios|rovermaster gate still broken (it merged 2 mins before zuul reported that) https://review.openstack.org/#/c/560445 so next run will be green07:49
ssbarneamarios|rover : i know after a ... funny, number of rechecks. that is why I was worried, as even if this gets +W right now, it will take a lot to get in.07:49
marios|roverssbarnea: ack on it gimme couple mins07:49
ssbarneasure, is not burning :D07:50
*** skramaja has joined #oooq07:51
*** tosky has joined #oooq07:52
quiquellssbarnea: Didn't know about (initial_scale_value | int)07:53
quiquellssbarnea: It's like a casting ?07:53
ssbarneaquiquell link please, in general i think i know how jinja filter work, yep using C terminology describes it well07:54
quiquellssbarnea: https://review.openstack.org/#/c/610501/3/roles/overcloud-scale/tasks/delete-original-node.yml07:54
*** fhubik has joined #oooq07:55
ssbarneaquiquell: I used round-parantheses to assure that the filter is applied in the right order, otherwise it can be confusing.07:55
quiquellssbarnea: ack07:56
quiquellmarios|rover: Do we have problems with nodepool ?07:56
ssbarneai have to confess that I do not know exactly the full priority list for operands, not sure exact where jinja  pipe "|" fits.07:56
ssbarneabut now that I read that line, it is a bit stupid :D07:57
ssbarneano real need to convert to int both operands, I think.07:57
marios|roverquiquell: well i saw a few more testenv issues this morning - we filed a but for it last night sec07:58
marios|rover    https://bugs.launchpad.net/tripleo/+bug/1797918 (infra related?) ovb jobs failing in run-v3.yaml with "(/home/zuul/src/git.openstack.org/openstack/tripleo-ci/toci_gate_test.sh:164): sudo kill -9 4733" Edit07:58
openstackLaunchpad bug 1797918 in tripleo "teclient returns failures when attempting to provision a stack in rdo-cloud" [Critical,Won't fix] - Assigned to Marios Andreou (marios-b)07:58
marios|roverquiquell: ^07:58
ssbarneabtw, i would recommend anyone to use "visual studio code + redhat yaml extension" as their editor, it is brilliant for ansible code.07:58
quiquellmarios|rover: ack07:58
quiquellssbarnea: interesting07:58
marios|roverquiquell: but weshay closed it (and see email thread about rdo outage) since it was resolved... maybe still issues07:58
ssbarneai had a plan to write a schema for zuul, did some POC but got blocked as JSON/YAML schema is not very easy to write.07:58
ssbarneai will do it during a weekend, end of sprint is not the time to work on that.07:59
quiquellssbarnea: How do we ensure that merging it does not break the gates again ?07:59
quiquellssbarnea: I mean new changes and all07:59
ssbarneaquiquell: zull will run it. if something new and bad goes in it will fail the gate and we have to rebase/fix, loosing n-hours.08:00
quiquellssbarnea: ack, let's keep rebasing this to be sure it's in the top08:01
ssbarneaquiquell: just get the votes on it, it would be more than enough.08:02
*** dtantsur|afk is now known as dtantsur08:03
ssbarneais not very common to have such errors, the syntax changes were made more than one year ago in ansible, so most people already know the official syntax.08:03
quiquellssbarnea: you got my +108:03
ssbarneathe broken centos ci jobs are so annoying and spammy, i wonder why we do not disable them until we can fix them.08:09
quiquellssbarnea: forget about them, remeber what panda|off said08:09
ssbarneain fact I am going to add a gmail filter for them "RedHat RDO CI has posted comments on this change" would go to trash.08:09
ssbarneais confusing because my top priority email forlder is the one with gerrit notifications.'08:10
ssbarneai remember someone had a special filter for gmail that was able to delete gerrit threads on merge or something similar.08:15
panda|offquiquell: ssbarnea I was wrong, those jobs are still important08:17
ssbarneapanda|off i guess, just less important than other ones08:18
quiquellpanda|off: so they are not going to rmeove phase1 or the like ?08:18
panda|offquiquell: nope08:18
quiquellpanda|off: ack08:18
ssbarneapanda|off : can you please +W this https://review.openstack.org/#/c/610038/ ? it failed to merge last night and lost the +W when i pressed rebase.08:19
*** gkadam has joined #oooq08:38
*** gkadam has quit IRC08:38
bogdandoo/ can we merge https://review.openstack.org/#/c/603069/ please08:39
bogdandoweshay: ^^08:39
*** skramaja has quit IRC08:44
*** skramaja has joined #oooq08:44
ssbarneaquiquell: please have a look at https://review.openstack.org/#/c/610018/ - i fixed problem reporte by sagi but I am curious if you think about sourcing in tripleo.sh08:46
bogdandoplease also let that thing in https://review.openstack.org/#/c/576746/08:46
ssbarneabogdando: regarding inject change, i am not sure if removing cloud-init is something we want.08:47
quiquellssbarnea: Found somethign regarding yaml08:47
quiquellmodule08:47
bogdandossbarnea: why not? you do not normally run it for non cloud images08:47
*** holser_ has joined #oooq08:47
bogdandoand that's the case, we want to install locally08:47
bogdandofor non cloud VMs*08:48
quiquellssbarnea: found issues08:49
ssbarneabogdando: you confused me, as long this image is run virtualized, i do see a reason for having cloud-init on it. I used cloud-init a lot for configuring images even without "clouds", just to give the initial settings.08:50
ssbarneafor example: why not using cloud-init script for the extra config? is not possible? just asking... not really an expert in the area.08:50
*** ykarel is now known as ykarel|mtg08:51
ssbarneai am asking this because uninstalling a default package is changing the testing base, which can have some undesired results later.08:51
ssbarneaquiquell: thanks, you really saved me a lot of time. fixed.08:53
bogdandoI think we need nothing cloud-* if running not in cloud08:55
bogdandoIMO)08:55
bogdandothere is nothing to init from, right?08:55
bogdandono metadata et al08:55
bogdandojust a libvirt vm08:56
bogdandosee also ssbarnea http://codesearch.openstack.org/?q=cloud-init&i=nope&files=&repos=tripleo-ci,tripleo-quickstart,tripleo-quickstart-extras08:56
bogdandoit's a pattern we have already08:57
*** chem has joined #oooq08:59
*** ykarel|mtg is now known as ykarel09:03
ssbarneabogdando: +1 it.09:09
ssbarneaany bindep expert around? how do I tell it to install libselinux-python on old systems and python3-libselinux on newer?09:09
ssbarneaplatform:rpm does not make any sense here09:10
ssbarneaalso even distro name does not because this is about distro version'09:10
quiquellssbarnea: Is my comment at your review correct ? https://review.openstack.org/#/c/61047509:11
quiquellssbarnea: I think that's why we are using ansible+python2 at zuul09:11
ssbarneaquiquell: yes, and yes, fixing it.09:11
quiquellssbarnea: cool !!!09:12
quiquell:-)09:12
quiquellssbarnea: This is the place where ansible is install, just calls install over setup.py of tripleo-quickstart09:14
ssbarneaquiquell: I think I found a trick for yum/dnf install: dnf install libselinux-python* python*-libselinux09:14
ssbarneathis command succeeds on both cases, as the behavior is to succeed if one package is found and installed09:15
quiquellssbarnea: no kidding...09:15
quiquellssbarnea: The list has priority ?09:15
*** panda|off has quit IRC09:15
ssbarneanope, but it does not matter, as our commands do already have excludes, so only one version would endup being installed.09:15
ssbarneawhat matters is to have both on the same line09:16
quiquellThis is going to be a pain to maintain09:16
ssbarneais a bit of a hack but avoids adding an extra conditional in many places09:16
ssbarneai do not see a problem with it, for now.09:16
*** fhubik is now known as fhubik|brb09:18
quiquellssbarnea: ping me when you have fixed https://review.openstack.org/#/c/61047509:18
ssbarneaquiquell the only hack is that DNF (not yum) needs a start after libselinux-python to avoid giving an error code,09:18
*** panda has joined #oooq09:18
ssbarneait makes some sense, yum accepts both so we are ok.09:18
ssbarneasure, few minutes to finish current one.09:19
*** sshnaidm|afk has joined #oooq09:32
ssbarneaquiquell: i updated https://review.openstack.org/#/c/610475/ - you can revisit it, still we will need to wait for results.09:43
quiquellssbarnea: Will exercise with the new job09:44
quiquellssbarnea: I suspect we are going to face a lot of issues with ansible + python309:45
ssbarneaquiquell: yes. but this is why we should keep it separated and slowly migrate towards pure py3. for the moment we should be ok with mixed.09:46
quiquellssbarnea: agree09:46
*** fhubik|brb has quit IRC09:52
*** saneax has quit IRC09:54
*** saneax has joined #oooq09:56
*** ykarel has quit IRC09:59
*** sshnaidm|afk is now known as sshnaidm10:03
*** ykarel has joined #oooq10:08
*** ykarel is now known as ykarel|lunch10:09
pandaany reviews to look at ?10:39
quiquellpanda: https://tree.taiga.io/project/tripleo-ci-board/task/153?kanban-status=144727610:41
pandaquiquell: on it10:42
*** ykarel_ has joined #oooq10:48
*** ykarel|lunch has quit IRC10:50
quiquellpanda:10:55
*** ykarel_ is now known as ykarel10:56
*** sshnaidm has quit IRC11:00
*** sshnaidm has joined #oooq11:01
*** quiquell is now known as quiquell|lunch11:01
pandassbarnea: did you find any place where we are still using tripleo.sh ?11:17
*** amoralej is now known as amoralej|lunch11:35
*** quiquell|lunch is now known as quiquell11:37
*** sshnaidm has quit IRC11:38
*** sshnaidm has joined #oooq11:38
ssbarneapanda: openstack-infra/tripleo-ci/... is full of references to that file.11:43
ssbarneahttp://codesearch.openstack.org/?q=tripleo%5C.sh&i=nope&files=&repos= also gives long list of results.11:45
sshnaidmssbarnea, when your fix or don't fix the issues in patches, please answer on every issue - if it's done, or if it's not done and why not done. Otherwise it's very difficult to track changes with rebases and new patchsets11:46
sshnaidmssbarnea, maybe in downstream it's different, but in upstream we work that way11:46
ssbarneasshnaidm: sure. it was not on purpose.11:54
ssbarneasshnaidm panda : can you please help with a bit of history/background regarding tripleo.sh?11:55
sshnaidmssbarnea, tripleo.sh was main script in CI 2 years before, in last sprints we removed its usage completely11:55
sshnaidmssbarnea, it's legacy non active code11:55
ssbarneaas part of the change I did grep for yum/python usage across the entire repository, so it did make sense to fix code that obviously would not run in new context.11:56
ssbarneado we have a reason for keeping the file there if is not needed?11:56
ssbarneaif not, I will not touch it and raise another CR to remove obsolete file.11:56
sshnaidmssbarnea, a good plan I think11:57
ssbarneasshnaidm what would be the updated version of this line? http://git.openstack.org/cgit/openstack-infra/tripleo-ci/tree/README.rst#n9311:59
ssbarneaor should I remove the entire section?11:59
*** ratailor has quit IRC12:00
pandawow, we should remove the entire file ...12:00
*** trown|outtypewww is now known as trown12:00
pandano information there is relevant anymore12:00
sshnaidmyeah, something like that12:00
sshnaidmthis file is misleading..12:00
pandathe only thing that makes sense is the punctuation12:01
pandacome commas and a full stop.12:01
pandasome*12:01
ssbarneai am trying to figure out the reason for POST_FAILURE on http://logs.openstack.org/38/610038/5/check/tripleo-ci-centos-7-containers-multinode/f3d6908/job-output.txt.gz12:04
sshnaidmpanda, I think removing of all legacy code deserves a user story12:04
ssbarneamarios|rover ^ looks like a rover stuff.12:04
ssbarneasshnaidm: please add one, i like to purge things. "cleansing" story.12:05
sshnaidmssbarnea, logs collection took more than hour (post timeout)12:05
sshnaidmssbarnea, I think only UA can12:05
ssbarneasshnaidm: but is this a reason to fail the job?12:06
sshnaidmssbarnea, yeah12:07
ssbarneai think is wrong to limit who can create story, but is up to UA to decide how the story gets priorietized (top or bottom)12:07
sshnaidmand you don't have logs, because logs collection failed..12:07
sshnaidmssbarnea, let panda clarify this12:07
ssbarneasshnaidm I seen at least 3 such failures in the last 24h, including in the linting fix that took almost 24h to merge. having this is killing out ability to ship (any)things;)12:08
quiquellGuys I don't feel well, I am going to rest, have put some stuff in the retro board12:10
ssbarneasshnaidm: should I create a gerrit bot that does recheck on watched patches? nobody will know is a bot, very useful for playing the game during nighttime12:10
*** quiquell is now known as quiquell|off12:11
*** dtrainor has joined #oooq12:14
pandasshnaidm: agreed, I'll create the use story and add a task from the issue we already have.12:15
pandassbarnea: everybody can physically create the user story12:16
pandassbarnea: what was agreed temporarily, and will be discussed at the retrospective is: 1) you add an epic only after discussing with the PM. 2) you add a user story to the sprint only after discussing it with me, because when something gets added, another thing may need to be removed.12:17
chandankumarpanda: weshay rfolco please have a look at this card https://tree.taiga.io/project/tripleo-ci-board/issue/16612:17
sshnaidmssbarnea, we have such bot12:18
sshnaidm%help12:19
sshnaidmwell, hubbot is sleeping.12:19
rfolcochandankumar, why this is an issue and not a task ?12:19
rfolcochandankumar, and it seems big enough to be a story12:20
pandachandankumar: also, why was this create after creating all the reviews ? :( it's difficult to balance the workload if the work are not tracked12:21
pandachandankumar: rfolco issue or story, this will go not go into sprint 20, because it ends today.12:21
chandankumarpanda: as discussed https://tree.taiga.io/project/tripleo-ci-board/issue/142?kanban-status=2027735 from this card , I need to fork to track seperatley12:21
ssbarneapanda marios sshnaidm : can you please recheck https://review.openstack.org/#/c/610038/ - i made a git-review mistake which I fixed (is the same as patch 5 which failed to mergE)12:22
*** hubbot1 has joined #oooq12:22
sshnaidmhubbot1, hi12:23
hubbot1sshnaidm: Error: "hi" is not a valid command.12:23
sshnaidm%status12:23
hubbot1sshnaidm: Error: "status" is not a valid command.12:23
sshnaidm%gatestatus12:23
hubbot1FAILING CHECK JOBS on master: tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset053, tripleo-ci-centos-7-scenario000-multinode-oooq-container-upgrades, tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset035 @ https://review.openstack.org/604298, stable/pike: tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset001, tripleo-ci-centos-7-ovb-1ctlr_1comp-featureset022 @ https://review.openstack.org/602248, master: tripleo-ci- (2 more messages)12:23
chandankumarrfolco: panda: what should i do now?12:23
pandachandankumar: chat with me12:24
pandachandankumar: if you have time12:24
chandankumarpanda: sure12:24
chandankumarpanda: bj?12:25
pandachandankumar: https://bluejeans.com/349250866912:25
ssbarneapanda can we workflow https://review.openstack.org/#/c/610501/ ? (rdo failures unrelated)12:27
ssbarneasshnaidm: please see https://review.openstack.org/#/c/610491/1/roles/validate-tempest/tasks/tempest-venv.yml@35 -- am I wrong to say is better to merge it like that? it works well.12:34
*** rlandy has joined #oooq12:35
*** sshnaidm has quit IRC12:36
ykarelis the issue known? seeing too many reds : https://review.rdoproject.org/zuul/status12:38
rlandyrascasoft: hi12:38
rascasoftrlandy, hey12:38
ykarelseeing failures related to testenv-brloker and copy logs(POST_FAILURES)12:38
ykarelmarios|rover, kopecmartin|ruck ^^12:38
rlandyrascasoft: testproject change kept failing on introspection ... did you get it to run further?12:38
rascasoftrlandy, yes, I included all the depends-on into the commit message and re run it12:39
rascasoftrlandy, the previous run failed because it was missing the inclusion of tripleo-ha-utils inside the toci script12:39
*** sshnaidm has joined #oooq12:39
rascasoftrlandy, so I included also that review12:39
rascasoftrlandy, and now it's running again12:40
rlandyrascasoft: ok - cool - then we are getting somewhere12:40
rlandyrascasoft: did the synchronize lines work?12:40
rascasoftrlandy, I honestly don't know12:40
rascasoftrlandy, we never get that far12:40
* rlandy checks12:40
rlandyrascasoft: really? I only see  the copy failure from yesterday and image prep failures12:43
rlandyhttps://review.rdoproject.org/zuul/builds?job_name=tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset04112:44
rascasoftrlandy, I'm following this https://review.rdoproject.org/zuul/stream/711d9fa0d6b942dba5dca75305ecd2af?logfile=console.log12:44
rascasoftbut at this point I'm not sure about anything12:44
rlandyrascasoft: hmm ... well that would be true if you stopped the jobs before the logs were created12:45
rlandyie: if you edited the job and rerun while logs were in process12:45
rascasoftrlandy, I honestly don't understand12:46
rlandyrascasoft: ok - well, let's juts watch this and see how it goes12:46
rlandyyou have all the patches included now12:46
rascasoftrlandy, yes, and by the way, using logic (because I don't have the knowledge) the fact that this job was kicked right after I changed the description with all the depends-on suggest me that at least this run is testing it12:47
rlandyit is12:47
rascasoftrlandy, so why you wrote about things being stuck since yesterday?12:48
ssbarneasshnaidm panda : does this explain why I am splitting arguments on separated lines https://s3.sbarnea.com/ss/181016-pre-tempest.yml__rdo_Workspace_.png12:49
rascasoftmarios|rover, kopecmartin|ruck, hey I've got a question: my assumption is that the docker_registry_host parameter is used by both undercloud and overcloud deploy commands, but for some reason, while deploying osp-14 I'm getting registry.access.redhat.com and I'm not able to identify where this value comes from12:51
marios|roverrascasoft: in call will check in bit12:51
rlandyrascasoft: stuck because the failure was before we could see your work12:57
rlandyweshay: https://review.rdoproject.org/r/#/c/16174/4/ci-scripts/infra-cleanup/ovb-tenant-cleanup.sh - comment here?12:58
ssbarneasshnaidm panda : i have a question related to DLRN tasks, in build-test-packages. Is the exclusive use of py3 really needed there? asking because changing code to work with py3 would require further changes. i see lots of calls made to pip or python which are likely to fail due to missing commands.12:59
sshnaidmssbarnea, it looks like bug in ansible linter, I hit the same in some cases13:00
sshnaidmssbarnea, it expects for object in various places when it's not really required13:00
ssbarneasshnaidm: please show me, i had the same impression but I found ways to avoid it.13:01
ssbarneaout logs were full of warnings about incorrect use of when, so I would not be surprised to see more failures.13:01
sshnaidmssbarnea, do you use vscode?13:03
rfolcopanda, is DoD a custom field of a task or just story ?13:05
*** sshnaidm is now known as sshnaidm|afk13:05
weshaykopecmartin|ruck, can you please jump into the master promotion jobs on https://review.rdoproject.org/zuul/status13:08
weshaykopecmartin|ruck, I see a lot of RED13:08
weshayping w/ questions13:09
*** sshnaidm|afk has quit IRC13:09
kopecmartin|ruckweshay, ok13:10
weshaymarios|rover, my only immediate to-do from that chat would be if you could review that cleanup script and patch13:11
mariosack weshay13:12
weshaykopecmartin|ruck, noting rdo-cloud is very unstable right now.. ping on irc13:12
weshayif you have any questions13:12
*** agopi has quit IRC13:15
kopecmartin|ruckweshay, i don't see there any promotion jobs, I see only check jobs failing13:15
mariosweshay: you mean https://review.rdoproject.org/r/#/c/16413/ already reviewed and rlandy had a -113:17
pandarfolco: just story13:17
pandarfolco: in task I put task requirements13:18
pandarfolco: but it's all up for debate13:18
pandarfolco: sorry, TEST requirements.13:18
pandarfolco: usually DOD in a task is standard "when the reviews are merged"13:19
rlandymarios: weshay; I'm confused https://review.rdoproject.org/r/#/c/16174/4/ci-scripts/infra-cleanup/ovb-tenant-cleanup.sh vs https://review.rdoproject.org/r/#/c/16413/13:19
rlandyI am -1 because I don;t think we can delete the instances in error13:19
rlandyhappy to be proved wrong here13:20
weshayrlandy, can't even openstack show $server_in_error13:20
weshayrlandy, so I'm w/ you13:20
weshayrlandy, we need to get into rhos-ops13:20
weshayand chat13:20
*** amoralej|lunch is now known as amoralej13:23
ssbarneapanda: is common_functions.sh (not j2 one) also part of the deprecation list, is consumed by toci_gate_test.sh scripts/deploy.sh scripts/tripleo.sh13:23
pandassbarnea: yes13:24
pandassbarnea: are you creating any CR ?13:24
pandaI'm promoting the cleanup card taht we have to a user story13:24
ssbarneaok, in this case i will avoid touching these files. nope, just updating https://review.openstack.org/#/c/61001813:24
pandassbarnea: and will create other user stories for clenaning up other parts13:25
pandassbarnea: ok thanks.13:25
ssbarneapanda: post link to story once you create it so I can link exiting CR to it.13:26
rlandythis is going nowhere ... https://review.rdoproject.org/zuul/stream/711d9fa0d6b942dba5dca75305ecd2af?logfile=console.log13:29
*** agopi has joined #oooq13:36
*** agopi_ has joined #oooq13:38
*** toure|gone is now known as toure13:38
*** agopi has quit IRC13:40
*** ykarel is now known as ykarel|afk13:50
*** vinaykns has joined #oooq13:53
weshayrfolco, panda did you guys review the status doc together?13:54
rfolcomore or less13:55
weshayrfolco, might as well use that as the update to the status for the tripleo meeting13:55
weshayjust ascii paste into the etherpad as well..13:55
weshayrfolco, we typically include a summary of.. http://rhos-release.virt.bos.redhat.com:3030/rhosp13:55
rfolcoweshay, hmmm but the ruck rover not set yet...13:55
weshayrfolco, /me adds to the doc to show u13:55
weshaymarios, kopecmartin|ruck ^13:56
* weshay gets13:56
rfolcoweshay, should we exclude ruck is ____ and rover is ____.... this will be decided tomorrow13:58
*** saneax has quit IRC13:59
rlandyrascasoft: this is stalled - https://review.rdoproject.org/zuul/stream/711d9fa0d6b942dba5dca75305ecd2af?logfile=console.log13:59
rlandycan I recheck it?13:59
weshay#tripleo meeting13:59
weshayrfolco, I updated14:02
weshaymarios, kopecmartin|ruck ^14:02
weshayhttps://docs.google.com/document/d/1XBDrbPyI-Cyl5mP0GhuuRDvsZteCwmXBaLFOZQaupU4/edit14:02
rfolcoweshay, I saw... was talking about this part14:02
rfolcoThe Ruck and Rover for this sprint are <ruck-person> (<nick_rover>) and <rover-person> (<nick_rover>). Please direct questions or queries to them regarding CI status or issues in #tripleo, ideally to whomever has the ‘|ruck’ suffix on their nick.14:02
rfolcoweshay, as we did not decide yet14:03
rfolcowe cannot simply point to the doc14:03
weshayno no14:03
weshayrfolco, ok.. just cut and paste out the relevant points to the etherpad for the tripleo status :)14:03
*** agopi_ is now known as agopi14:04
rlandyugh - we really need to update the reproducer :(14:04
rlandyno way to get by ironic errors atm14:04
weshayok guys...14:04
weshaymove to #tripleo please14:04
rfolco!gatestatus14:09
openstackrfolco: Error: "gatestatus" is not a valid command.14:09
rfolco%help14:09
hubbot1rfolco: (help [<plugin>] [<command>]) -- This command gives a useful description of what <command> does. <plugin> is only necessary if the command is in more than one plugin. You may also want to use the 'list' command to list all available plugins and commands.14:09
rascasoftrlandy, whatever is good for you14:10
rlandy rechecked already14:11
weshayhttps://review.openstack.org/#/c/582294/14:12
rfolcokopecmartin|ruck, can you please add one line status update to https://etherpad.openstack.org/p/tripleo-ci-squad-meeting - L3914:12
kopecmartin|ruckrfolco, sure14:13
rfolcokopecmartin|ruck, thx14:13
weshayhttps://review.openstack.org/#/c/579603/14:16
weshay! gatestatus14:17
openstackweshay: Error: "gatestatus" is not a valid command.14:17
weshaymeh14:17
ssbarneapanda: if you can recast W on https://review.openstack.org/#/c/610038/ it would great.14:19
rlandy%gatestatus14:21
hubbot1FAILING CHECK JOBS on master: tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset053, tripleo-ci-centos-7-scenario000-multinode-oooq-container-upgrades, tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset035 @ https://review.openstack.org/604298, stable/pike: tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset001, tripleo-ci-centos-7-ovb-1ctlr_1comp-featureset022 @ https://review.openstack.org/602248, master: tripleo-ci- (2 more messages)14:21
rlandyweshay: ^^14:22
hubbot1FAILING CHECK JOBS on master: tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset053, tripleo-ci-centos-7-scenario000-multinode-oooq-container-upgrades, tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset035 @ https://review.openstack.org/604298, stable/pike: tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset001, tripleo-ci-centos-7-ovb-1ctlr_1comp-featureset022 @ https://review.openstack.org/602248, master: tripleo-ci- (2 more messages)14:22
weshayrlandy, you follow #tripleo14:23
weshayinteresting14:23
rlandyweshay: I am14:25
rlandyweshay: personally, I agree we should fix the errors - not avoid them14:26
rlandyjust my $0.0214:26
rlandymy vote is with dtantsur, "OVB is an openstack cloud. if we cannot make it reliable enough for us.. we're in trouble"14:27
weshaysay it in #tripleo14:27
weshayMY LANTA14:27
weshayFOLKS14:27
weshayGET IN THE MEETING14:27
rlandywe have moved on14:27
pandassbarnea: why did you remove the changes in the j2 in https://review.openstack.org/#/c/610038/ ?14:29
*** ykarel|afk has quit IRC14:29
* dtantsur tries really hard not to suggest replacing ovb with containers. really hard.14:29
chandankumarpanda: can you add ykarel|afk https://tree.taiga.io/project/tripleo-ci-board/ to the board so that he can add the user story14:30
*** ykarel|afk has joined #oooq14:30
chandankumarrfolco: weshay ykarel@redhat.com14:32
chandankumarplease add him so that he can dd the user stories14:32
chandankumarAdd Epic option is not visible to him14:33
pandachandankumar: what epic needs to be added ?14:38
chandankumarpanda: ykarel|afk is asking for python3 porting of tempest plugins14:39
chandankumarit needs to be added to epic first then reviewed by pm14:39
pandachandankumar: was this discussed with PM ? yatin would be an external member, I can't activate epic addition for external member14:40
chandankumarweshay: ^^14:40
ykarel|afkpanda, no issue for me if someone else can create and get it completed14:41
ykarel|afkmain idea is to complete converision of project, mostly those are done, mainly remaining are tempest plugins and some neutron projects:- https://review.rdoproject.org/etherpad/p/py3_conversion_status14:44
ykarel|afkearlier around 2-3 weeks ago it was told that it will be done in next sprint, may be a card was created that time14:45
ykarel|afkbut i don't know how it is being tracked/worked upon14:45
*** ykarel|afk has quit IRC14:48
*** ykarel|afk has joined #oooq14:48
weshaykopecmartin|ruck, what have you found in master?14:55
*** ccamacho has quit IRC14:57
weshayrfolco, I jumped the gun there14:57
weshaysorry14:57
weshayat the end of the meeting post it14:58
weshay:)14:58
rfolconot too bad14:58
kopecmartin|ruckweshay, nothing specific yet, are we talking about tripleo promotion jobs?14:59
*** skramaja has quit IRC14:59
*** ccamacho has joined #oooq15:00
weshaykopecmartin|ruck, yes15:00
weshaykopecmartin|ruck, for master15:01
weshayshould be fairly straight forward afaict15:01
pandarfolco: you have the TC meeting now too ?15:02
kopecmartin|ruckweshay, it seems most of them are failing with this https://logs.rdoproject.org/openstack-periodic/git.openstack.org/openstack-infra/tripleo-ci/master/periodic-tripleo-ci-centos-7-multinode-1ctlr-featureset017-master/275de86/logs/undercloud/home/zuul/install-undercloud.log.txt.gz#_2018-10-16_11_39_14_76615:06
weshaykopecmartin|ruck, k.. check to see if there is a bug on that, if not please open and set as a promotion-blocker ( alert is not needed )15:07
kopecmartin|ruckweshay, already on it15:07
weshaykopecmartin++15:10
chandankumarsee ya guys tomorrrow15:19
*** chandankumar is now known as chkumar|off15:19
kopecmartin|ruckweshay, https://bugs.launchpad.net/tripleo/+bug/179812615:25
openstackLaunchpad bug 1798126 in tripleo "Undercloud deploy - 'NoneType' object has no attribute 'group'" [Critical,Triaged]15:25
kopecmartin|ruckweshay, this job has no log, is it related to ^^ bug or is it something different? https://logs.rdoproject.org/openstack-periodic/git.openstack.org/openstack-infra/tripleo-ci/master/periodic-tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset035-master/236ae5e/15:28
*** holser_ has quit IRC15:29
pandarascasoft: forget about the suggetion to change "any" it's actually more useful to have ti in any for a while, to check if the right packages are arriving on the wrong interface.15:31
rascasoftpanda, yeah, now that you're saying that I recall we decided to keep any exactly for this at that time15:32
*** sshnaidm has joined #oooq15:37
*** hamzy has quit IRC15:42
pandarlandy: whats' the status here https://tree.taiga.io/project/tripleo-ci-board/issue/141?kanban-status=2027734 ? are you able to complete this today ?16:01
*** sshnaidm has quit IRC16:13
*** quiquell|off has quit IRC16:15
rlandypanda: depends if we can get the ovb job to run past introspection to actually test ha-utils16:17
rlandytrying to get this killed ... https://review.rdoproject.org/zuul/stream/711d9fa0d6b942dba5dca75305ecd2af?logfile=console.log16:18
rlandyit's stuck16:18
*** ykarel|afk is now known as ykarel16:18
*** apetrich has quit IRC16:18
rlandypanda: either way, reviews are all there - but we need to test out16:19
*** d0ugal has quit IRC16:21
*** holser_ has joined #oooq16:21
pandarlandy: I still see 5 unmerged reviews. Doesn't seem realistic to have those merged today. If it's blocked, and you want to work on the preparation of sprint 21, it may be more healthy to detach from the sprint, and eventually reattach to the next16:21
rlandypanda: sure - whatever helps planning16:22
hubbot1FAILING CHECK JOBS on master: tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset053, tripleo-ci-centos-7-scenario000-multinode-oooq-container-upgrades, tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset035 @ https://review.openstack.org/604298, stable/pike: tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset001, tripleo-ci-centos-7-ovb-1ctlr_1comp-featureset022 @ https://review.openstack.org/602248, master: tripleo-ci- (2 more messages)16:22
pandaok, detaching16:22
*** d0ugal has joined #oooq16:23
pandakopecmartin|ruck: any progress on https://tree.taiga.io/project/tripleo-ci-board/issue/27?kanban-status=2027734 ?16:28
*** apetrich has joined #oooq16:30
* rlandy needs to run out for a bit - back in an hour or so16:31
*** rlandy is now known as rlandy|biab16:32
*** ykarel is now known as ykarel|away16:37
kopecmartin|ruckpanda, not yet :/16:43
pandakopecmartin|ruck: ok, detaching from sprint 20, feel free to attach it to sprint 2116:44
*** bogdando has quit IRC16:44
*** ykarel|away has quit IRC16:46
kopecmartin|ruckpanda, ok, thanks16:46
*** panda is now known as panda|off16:48
*** chkumar|off is now known as chandankumar16:57
*** dtantsur is now known as dtantsur|afk17:03
*** kopecmartin|ruck is now known as kopecmartin|off17:05
*** hamzy has joined #oooq17:05
*** panda|off has quit IRC17:24
*** panda has joined #oooq17:27
*** amoralej is now known as amoralej|off17:28
*** vinaykns has quit IRC17:30
rfolcoweshay, ssbarnea can you both please log in to https://softwarefactory.usersys.redhat.com with your redhat email account ?17:32
weshayredhat or github?17:33
rfolcoweshay, hmm true not sure it allows redhat directly17:33
weshaygithub worked17:33
rfolcok17:34
rfolcoweshay, feel free to change your preferred email in the settings17:34
rfolcok will add sorin later then17:35
weshayI'm in.. shiny17:35
ssbarnearfolco: github worked. i am not going to try my corp17:35
rfolcossbarnea, k, trying again to add you as a member17:36
*** trown is now known as trown|lunch17:37
*** rlandy|biab is now known as rlandy17:46
ssbarnearfolco weshay rlandy : https://review.openstack.org/#/c/610038/ please, before you leave (so hopefully by tomorrow it would merge)17:54
*** jfrancoa has quit IRC17:59
rlandyweshay: ping re: centos version for queens, rocky, master18:10
rlandyCentOS-7-x86_64-GenericCloud-170618:10
rlandy CentOS-7-x86_64-GenericCloud-170718:11
rlandyCentOS-7-x86_64-GenericCloud-170818:11
rlandyCentOS-7-x86_64-GenericCloud-171018:11
rlandyCentOS-7-x86_64-GenericCloud-171118:11
rlandypick a winner18:11
chandankumarssbarnea: weshay rlandy I need some help here https://review.openstack.org/#/c/609666/4/playbooks/tripleo-ci/run-v3.yaml@5718:11
chandankumarI am not sure how to put that18:11
chandankumarwill I use set_facts and add var by setting default values?18:13
rlandychandankumar: you could use https://github.com/openstack-infra/tripleo-ci/blob/master/playbooks/tripleo-ci/vars/common.yaml18:14
rlandybut that is not at task level18:14
rlandyit would be a var though18:14
rlandyalthough that does look like the place where the var read in the run-v3 playbook are18:15
rlandyreally it's a stylistic comment18:16
rlandynothing blocking18:16
chandankumarrlandy: will i create a tempest specific vars?18:16
rlandyyou could try that vars file18:16
chandankumarand put it in a seperate file IN future it is going to change as we move os_tempest role18:16
rlandywell, in this case, looking at the task at hand18:16
rlandythe var would be overridable_settings_list18:17
*** vinaykns has joined #oooq18:17
rlandywhich in time, could be more than tempest18:17
chandankumarrlandy: yes,18:17
rlandyin this case, it relates to what we allow to be overridden18:17
rlandytempset is just the only acceptable case as of now18:18
chandankumarrlandy: I am thinking we can add some script, if we set fs override then for a particular fs it shoudl export all vars and reuse that18:19
chandankumarinstead of checking in a list18:19
rlandychandankumar: yep - sounds reasonable ...18:19
rlandyrfolco: ^^18:19
rlandythat was your work, right?18:19
* chandankumar sorry crazy ideas flows from my mind18:19
rlandychandankumar: for the moment, I would just go for the list var18:20
rlandyand we can reconsider next sprint18:20
chandankumarrlandy: sure, I will updte the patch in the monring18:20
rlandylet's male panda happy and close something out :)18:20
rfolcorlandy, chandankumar: we had put on hold the fs override, the team did not get any conclusion on how we should expose it18:20
rlandymake18:20
chandankumarrfolco: https://review.openstack.org/#/q/topic:tempest_standalone+(status:open+OR+status:merged)18:21
chandankumarrfolco: we are consuming happily here to do some awesome stuff18:21
* rlandy goes back to upshift18:22
hubbot1FAILING CHECK JOBS on master: tripleo-ci-centos-7-scenario000-multinode-oooq-container-upgrades, tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset001, tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset053, tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset035 @ https://review.openstack.org/604298, stable/pike: tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset001, tripleo-ci-centos-7-ovb-1ctlr_1comp_1ceph-featureset024,  (3 more messages)18:22
* chandankumar hides back18:23
*** hamzy has quit IRC18:23
rfolcochandankumar, ah yeah tempest only configs18:24
rfolcoyou mentioned featureset18:24
chandankumarrfolco: in future, we may extend fs standalone to deploy only service specific stuff that would be useful18:24
rfolcok sounds reasonable18:24
chandankumarin order to gate multiple projects18:25
rlandyweshay: ping me - again - we need a common user18:50
rlandyfor ci18:50
rlandythat can join out LDAP rover group18:50
weshayrlandy, are you getting pinged about that?18:51
weshayor is that for upshift18:51
weshayrlandy, did we get a user for jenkins at somepoint?18:51
rlandyweshay: I am not getting, I am pinging you :)18:51
rlandyweshay: correct for upshift18:52
weshaycan I see the requirement18:52
rlandybecause I log in with my username and password18:52
rlandyI don't want that on the slave shared18:52
rlandyI can test with my password for now18:52
rlandybut we need something shared for general CI use18:52
weshaysure18:52
rlandywhat was that user for jenkins you mentioned?18:53
rlandyoh the rhos-jenkins user?18:54
rlandyweshay: ^^ that user?18:56
weshaysomething like that /me checks notes18:56
rlandycan we add that user to the rover group?18:56
rlandyalso we'd need a password for our user18:56
*** ssbarnea_ has joined #oooq18:56
weshayor upstream-redhat-rdo-ci18:57
weshayrlandy, k.. give me a min18:57
rlandysure - take your time - doing a flavor size check atm18:58
ssbarnea_weshay: i see very often timeout failures from zuul, https://review.openstack.org/#/c/610018/ -- any way to avoid this?19:03
weshayssbarnea_, yes.. move to a standalone deployment19:04
weshayeducate engineers about infra19:04
weshayinclude ara such that it captures overcloud tasks19:04
weshayand track results in graphana19:04
weshayssbarnea_, welcome to the party :)19:05
*** trown|lunch is now known as trown19:07
weshayssbarnea_, make sense?19:08
ssbarnea_partially. considering that that failed job had multinode in its name, I don't see how standalone would cover the same use-case.19:10
ssbarnea_i am infra meeting, so i will reply slower here.19:10
*** holser_ has quit IRC19:16
weshayrlandy, I'm not able to add service accounts to rover groups19:23
weshayssbarnea_, no worries19:24
rlandyweshay: ok - I am having trouble accessing the tenant anyways ...19:24
rlandywill need to ask19:24
rlandyMax retries exceeded with url: /v3/auth/tokens (Caused by SSLError(SSLError("bad handshake: Error([('SSL routines', 'tls_process_server_certificate', 'certificate verify failed')],)",),))19:24
rlandy^^ that for the v3 file19:24
weshayrlandy, fyi https://docs.google.com/document/d/11n102A61Hm9swSX0Vgfs2YDuuO30G9E-9_mN99dGfyI/edit#heading=h.8vgafc54hz3h19:24
weshaynot sure if that will work though19:25
weshayrlandy, oh do we need the redhat cert on the box where it's running?19:25
rlandyI think so19:25
rlandyfor v219:25
rlandyv319:25
rlandyfor v2 ...19:25
rlandyFailed to discover available identity versions when contacting https://rhos-ocp.infra.prod.upshift.eng.rdu2.redhat.com:13000. Attempting to parse version from URL.19:25
ssbarnea_this is why i think we are better of using a non corp domain with a valid SSL certificate that we can get from letsencrypt. what do you think?19:28
rlandytosky: hello19:54
toskyrlandy: hi19:54
*** hamzy has joined #oooq19:54
rlandytosky: does your team use the new rhos upshift , by any chance?19:55
toskynot yet, I think19:55
rlandyok - long  shot :)19:56
hubbot1FAILING CHECK JOBS on master: tripleo-ci-centos-7-scenario000-multinode-oooq-container-upgrades, tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset001, tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset053, tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset035 @ https://review.openstack.org/604298, stable/pike: tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset001, tripleo-ci-centos-7-ovb-1ctlr_1comp_1ceph-featureset024,  (3 more messages)20:22
*** holser_ has joined #oooq20:41
*** ssbarnea has quit IRC20:42
*** ssbarne__ has joined #oooq20:42
*** ssbarnea has joined #oooq20:42
*** hamzy has quit IRC20:44
*** ssbarnea_ has quit IRC20:44
*** ssbarne__ has quit IRC20:57
*** sshnaidm has joined #oooq21:01
rlandyOMG21:01
rlandyweshay++21:01
hubbot1rlandy: weshay's karma is now 1121:01
rlandywe're in21:01
*** sshnaidm is now known as sshnaidm|off21:01
*** trown is now known as trown|outtypewww21:04
*** holser_ has quit IRC21:06
*** toure is now known as toure|gone21:31
*** sshnaidm|off has quit IRC21:48
hubbot1FAILING CHECK JOBS on master: tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset001, tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset053, tripleo-ci-centos-7-scenario000-multinode-oooq-container-upgrades, tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset035 @ https://review.openstack.org/604298, stable/pike: tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset001, tripleo-ci-centos-7-ovb-1ctlr_1comp-featureset022, tripleo-ci- (3 more messages)22:22
*** chem has quit IRC22:38
*** agopi is now known as agopi|brb22:52
*** vinaykns has quit IRC22:54
*** agopi|brb has quit IRC22:56
*** tosky has quit IRC23:10
*** hamzy has joined #oooq23:24
*** ssbarnea|bkp2 has joined #oooq23:46
*** ssbarnea has quit IRC23:46

Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!