Wednesday, 2014-03-05

*** aignatov has quit IRC00:04
*** aignatov has joined #openstack-ironic00:06
Haomenghello ironic, who know such issue '2014-03-05 08:00:09.026 | sudo: cp_it: command not found' during run devstack following these steps - https://etherpad.openstack.org/p/IronicAndDevstackAgain00:07
openstackgerritLucas Alvares Gomes proposed a change to openstack/ironic: do_node_deploy() to use greenthreads  https://review.openstack.org/7804700:07
NobodyCamHaomeng: I have not run into that my self but have heard changing cp_it to co fixes it00:08
NobodyCamoh and good morning00:08
HaomengI look into the devstack scripts,  cp_it is a function method in our script ./functions-common, but not sure why it can not be found00:09
HaomengNobodyCam: good evening:)00:09
NobodyCam:)00:09
HaomengNobodyCam: should we commit new patch to devstack to fix our ironic script cp_it issue?00:09
NobodyCamHaomeng: I have not looked into that issue yet00:10
NobodyCambut yes if you can see a fix00:10
HaomengNobodyCam: I want to have a try with my machine, how do you think if only physical network adapter is enough for our devstack+ironic?00:10
HaomengNobodyCam: ok, let me check the root cause, and try to fix00:10
HaomengNobodyCam: my pc has only one physical network card00:11
NobodyCamHaomeng: I think one adapter should work with devstack00:11
HaomengNobodyCam: ok, thank you:)00:11
HaomengNobodyCam: enjoy your evening and night:)00:11
HaomengNobodyCam: :)00:11
NobodyCamThank you Haomeng :)00:11
HaomengNobodyCam: welcome:)00:11
*** jdob has quit IRC00:25
*** adam_g has joined #openstack-ironic00:27
ShrewsNobodyCam, Haomeng: interesting that neither NobodyCam nor myself see that error, but Haomeng and adam_g do. Weird.00:32
NobodyCamShrews: I test with dib not devstack00:33
adam_gdevstack is doing 'sudo cp_it' and losing cp_it from its env, AFAICS00:34
NobodyCamoh root pathing issue?00:35
Haomengadam_g: cp_it is defined by functions script00:35
Haomenghowever our ironic script did not not source $TOP/functions00:36
NobodyCamdoh00:36
Haomeng:)00:37
adam_gHaomeng: first tmie looking at devstack in a while but i thought that was sourced elsewhere. i was able to work around by chown'ing /tftpboot first, then removing sudo from tha tcall00:40
Haomengadam_g: yes, that is confusing why it can not find the cp_it function, I can recreate this issue with two different computer00:41
Haomengadam_g: the system I used is a refresh ubuntu that is new installation, and I follow the steps one by one00:42
adam_gHaomeng: sudo apt-get update00:42
adam_gsudo apt-get -y upgrade00:42
adam_gsudo add-apt-repository cloud-archive:havana00:42
adam_gsudo apt-get update00:42
adam_gOops00:42
adam_gHaomeng: http://paste.ubuntu.com/7036006/ is what i meant00:43
Haomengyes, I did all steps in this guide - https://etherpad.openstack.org/p/IronicAndDevstackAgain00:44
Haomengadam_g: ok, let me check http://paste.ubuntu.com/7036006/00:44
Haomengadam_g: not sure if it is same one with https://etherpad.openstack.org/p/IronicAndDevstackAgain00:44
Haomengadam_g: got it, you mean we did not use cp_it now00:45
Haomengadam_g: "+    sudo chown -R $STACK_USER:$LIBVIRT_GROUP /tftpboot" is new line00:45
Haomengadam_g: but we still call cp_it00:45
Haomengthe question is where we source/include functions to call cp_it00:46
Haomengwill be away for a while...00:49
Haomengadam_g: I run stack with non-root user, not sure if it is sudo issue00:50
adam_gHaomeng: stack.sh -> functions -> functions-common00:52
Haomengadam_g: yes, but not sure why cp_it can not be found00:53
adam_ghttp://paste.ubuntu.com/7036051/00:54
Haomenglet me try again with othere ubuntu sys00:54
lucasagomesmrda, hey I noticed u were working in a bug #1285507, I didn't see this bug and opened another one today about it today #1287979 (which now is duplicated)00:55
lucasagomesdo you already have any patches ready for those fix?00:56
Haomenghttp://paste.openstack.org/show/72385/00:56
Haomengshould be sudo issue00:56
mrdalucasagomes: hang on :)00:56
Haomengsudo will fork new process?00:56
lucasagomesmrda, I submitted a patch for the do_node_deploy() (https://review.openstack.org/#/c/78047/) but if u already have something ready I'm glad to abandon mine00:56
lucasagomesmrda, ack00:57
Haomengadam_g:  http://paste.openstack.org/show/72385/00:57
*** Hefeweizen has joined #openstack-ironic00:58
Haomengadam_g: we should not use sudo I think00:58
*** matty_dubs|sfo is now known as matty_dubs|gone00:59
Haomengadam_g: other scripts calling cp_it, they did not run as 'sudo cp_it'00:59
mrdalucasagomes: So I do have some code for +bug/1285507 but it's not ready.  I'm having trouble with save_and_reraise_exception() not doing what I want.  If you have a patch that's good to go, feel free to take over this problem01:00
adam_gHaomeng: yeah, that diff i pasted is what i used to avoid it and workaround for the time being01:00
lucasagomesmrda, right, sorry I didn't see that bug until rloo point me to that01:00
mrdalucasagomes: i.e. I'm trying to propagate the exception out of the _spawn'ed task, but it's not working :)01:01
lucasagomesmrda, that patch only fix the do_node_deploy() the do_node_tear_down() still needs work01:01
lucasagomesmrda, why u would need that?01:01
lucasagomesmrda, for async tasks, u can just add the error to the nodes last_error field01:01
*** yongli has joined #openstack-ironic01:01
mrdaI didn't want to change the public behaviour for the do_node_deploy01:02
adam_gHaomeng: oh, looks like the patch has been udpated in gerrit to fix that01:03
mrdathere's a few tests that check for the exception raised and I wanted to maintain that behaviour01:03
mrdalucasagomes: ok, I see your review01:03
mrdalucasagomes: how about I reference your fix in my bug, and I'll add code for the tear_down following the style of your patch?01:04
lucasagomesmrda, sure, that would work01:04
lucasagomesI will change that Partial-Bug flag01:04
mrdaok, thanks lucasagomes01:05
*** blamar has quit IRC01:05
lucasagomesmrda, thank u, and sorry for stepping on ur toes01:05
openstackgerritLucas Alvares Gomes proposed a change to openstack/ironic: do_node_deploy() to use greenthreads  https://review.openstack.org/7804701:06
mrdathat's ok lucasagomes , just so long as the bugs get fixed :)01:06
lucasagomes:)01:07
mrdalucasagomes: I might lean on you for review for the bug fix regarding XML and WSME that I'm working on01:08
mrda(as penance)01:08
mrda:)01:08
lucasagomesmrda, hah deal01:08
*** derekh has quit IRC01:24
*** rwsu has quit IRC01:29
*** harlowja has quit IRC01:31
openstackgerritLucas Alvares Gomes proposed a change to openstack/ironic: deploy() to fail if image is larger than root partition  https://review.openstack.org/7800101:31
*** rloo has quit IRC01:32
*** lucasagomes has quit IRC01:32
*** vkozhukalov has joined #openstack-ironic01:36
*** vkozhukalov has quit IRC01:42
Haomengadam_g: ok, let me check the new patch out, thanks01:45
Haomengadam_g: I workaround the cp_it issue, and encoutered new issue - "2014-03-05 09:00:18.646 | nova-compute: no process found", do you have any idea?01:51
*** rloo has joined #openstack-ironic01:52
openstackgerritA change was merged to openstack/ironic: Sync the log_handler from oslo  https://review.openstack.org/7650001:55
*** nosnos has joined #openstack-ironic01:56
openstackgerritA change was merged to openstack/ironic: Fix SeaMicro driver post landing for ba207b4aa0  https://review.openstack.org/7773901:57
*** harlowja has joined #openstack-ironic02:17
*** jcooley_ has quit IRC02:36
*** coolsvap1 has quit IRC02:55
*** harlowja is now known as harlowja_away03:13
NobodyCamahh only an hour and 45 minutes to go 24 miles03:17
*** rloo_ has joined #openstack-ironic03:17
*** rloo has quit IRC03:21
*** rloo_ has quit IRC03:26
ShrewsNobodyCam: ugh03:29
*** Shrews has quit IRC03:32
*** Shrews has joined #openstack-ironic03:32
HaomengNobodyCam: :)03:36
NobodyCam:)03:56
NobodyCam:-p03:56
NobodyCamrum and coke makes things better03:56
*** harlowja_away has quit IRC04:17
*** rloo has joined #openstack-ironic04:21
*** rloo has quit IRC04:24
*** rloo has joined #openstack-ironic04:25
*** killer_prince2 has joined #openstack-ironic04:29
*** jcooley_ has joined #openstack-ironic05:36
*** jcooley_ has quit IRC05:40
*** jcooley_ has joined #openstack-ironic05:42
*** rloo has quit IRC06:03
openstackgerritJenkins proposed a change to openstack/ironic: Imported Translations from Transifex  https://review.openstack.org/7119206:06
*** loki_ has joined #openstack-ironic06:10
*** romcheg has joined #openstack-ironic06:21
*** jcooley_ has quit IRC06:22
*** romcheg has quit IRC06:23
*** jcooley_ has joined #openstack-ironic06:38
*** vkozhukalov has joined #openstack-ironic06:47
*** mrda is now known as mrda_away06:50
*** jcooley_ has quit IRC06:50
*** jcooley_ has joined #openstack-ironic06:51
*** Manishanker has joined #openstack-ironic06:52
*** jcooley_ has quit IRC06:56
*** vkozhukalov has quit IRC07:00
*** saju_m has joined #openstack-ironic07:02
*** jcooley_ has joined #openstack-ironic07:03
*** lsmola has joined #openstack-ironic07:04
*** jcooley_ has quit IRC07:16
*** jcooley_ has joined #openstack-ironic07:16
*** jcooley_ has quit IRC07:21
*** max_lobur has joined #openstack-ironic07:46
*** ifarkas has joined #openstack-ironic07:50
*** romcheg has joined #openstack-ironic07:53
*** jcooley_ has joined #openstack-ironic07:53
*** jcooley_ has quit IRC07:57
*** nosnos has quit IRC08:11
*** openstackgerrit has quit IRC08:11
*** antonym has quit IRC08:11
*** ewindisch has quit IRC08:11
*** saju_m has quit IRC08:11
*** Shrews has quit IRC08:11
*** aignatov has quit IRC08:11
*** Lord_Set has quit IRC08:11
*** shortstop has quit IRC08:11
*** SpamapS has quit IRC08:11
*** dhellmann_ has quit IRC08:11
*** pradipta has quit IRC08:11
*** yongli has quit IRC08:11
*** krtaylor has quit IRC08:11
*** killer_prince has quit IRC08:11
*** morgabra has quit IRC08:11
*** sbadia has quit IRC08:11
*** russell_h has quit IRC08:11
*** bigjools has quit IRC08:11
*** lsmola has quit IRC08:11
*** jgrimm has quit IRC08:11
*** Madasi has quit IRC08:11
*** rainya has quit IRC08:11
*** mdurnosvistov has quit IRC08:11
*** mrda_away has quit IRC08:11
*** davidlenwell has quit IRC08:11
*** lynxman has quit IRC08:11
*** jrist has quit IRC08:11
*** greghaynes has quit IRC08:11
*** ifarkas has quit IRC08:11
*** max_lobur has quit IRC08:11
*** Hefeweizen has quit IRC08:11
*** pleia2 has quit IRC08:11
*** comstud has quit IRC08:11
*** early has quit IRC08:11
*** matty_dubs|gone has quit IRC08:11
*** lifeless has quit IRC08:11
*** GheRivero has quit IRC08:11
*** mordred has quit IRC08:11
*** JayF has quit IRC08:11
*** EmilienM has quit IRC08:11
*** zigo has quit IRC08:11
*** jroll has quit IRC08:11
*** wendar has quit IRC08:11
*** russellb has quit IRC08:11
*** adam_g has quit IRC08:11
*** mikal has quit IRC08:11
*** Manishanker has quit IRC08:11
*** loki_ has quit IRC08:11
*** Haomeng has quit IRC08:11
*** ekarlso has quit IRC08:11
*** pquerna has quit IRC08:11
*** JoshNang_ has quit IRC08:11
*** anteaya has quit IRC08:11
*** NobodyCam has quit IRC08:11
*** devananda has quit IRC08:11
*** nosnos has joined #openstack-ironic08:28
*** nosnos has quit IRC08:29
*** yuriyz has joined #openstack-ironic08:43
*** yuriyz has quit IRC08:48
*** jistr has joined #openstack-ironic08:49
*** yuriyz has joined #openstack-ironic08:50
*** max_lobur has joined #openstack-ironic09:01
*** ndipanov has joined #openstack-ironic09:01
*** openstackgerrit has joined #openstack-ironic09:01
*** Manishanker_ has joined #openstack-ironic09:01
*** antonym has joined #openstack-ironic09:01
*** ewindisch has joined #openstack-ironic09:01
*** nosnos_ has joined #openstack-ironic09:01
*** ifarkas has joined #openstack-ironic09:01
*** lsmola has joined #openstack-ironic09:01
*** saju_m has joined #openstack-ironic09:01
*** Manishanker has joined #openstack-ironic09:01
*** loki_ has joined #openstack-ironic09:01
*** Shrews has joined #openstack-ironic09:01
*** yongli has joined #openstack-ironic09:01
*** Hefeweizen has joined #openstack-ironic09:01
*** adam_g has joined #openstack-ironic09:01
*** aignatov has joined #openstack-ironic09:01
*** jgrimm has joined #openstack-ironic09:01
*** krtaylor has joined #openstack-ironic09:01
*** ekarlso has joined #openstack-ironic09:01
*** killer_prince has joined #openstack-ironic09:01
*** rainya has joined #openstack-ironic09:01
*** greghaynes has joined #openstack-ironic09:01
*** Madasi has joined #openstack-ironic09:01
*** Lord_Set has joined #openstack-ironic09:01
*** pleia2 has joined #openstack-ironic09:01
*** comstud has joined #openstack-ironic09:01
*** Haomeng has joined #openstack-ironic09:01
*** mdurnosvistov has joined #openstack-ironic09:01
*** mrda_away has joined #openstack-ironic09:01
*** pquerna has joined #openstack-ironic09:01
*** davidlenwell has joined #openstack-ironic09:01
*** JoshNang_ has joined #openstack-ironic09:01
*** shortstop has joined #openstack-ironic09:01
*** SpamapS has joined #openstack-ironic09:01
*** morgabra has joined #openstack-ironic09:01
*** jrist has joined #openstack-ironic09:01
*** dhellmann_ has joined #openstack-ironic09:01
*** sbadia has joined #openstack-ironic09:01
*** pradipta has joined #openstack-ironic09:01
*** lynxman has joined #openstack-ironic09:01
*** bigjools has joined #openstack-ironic09:01
*** russell_h has joined #openstack-ironic09:01
*** JayF has joined #openstack-ironic09:01
*** EmilienM has joined #openstack-ironic09:01
*** anteaya has joined #openstack-ironic09:01
*** early has joined #openstack-ironic09:01
*** matty_dubs|gone has joined #openstack-ironic09:01
*** lifeless has joined #openstack-ironic09:01
*** GheRivero has joined #openstack-ironic09:01
*** mordred has joined #openstack-ironic09:01
*** zigo has joined #openstack-ironic09:01
*** jroll has joined #openstack-ironic09:01
*** wendar has joined #openstack-ironic09:01
*** russellb has joined #openstack-ironic09:01
*** mikal has joined #openstack-ironic09:01
*** NobodyCam has joined #openstack-ironic09:01
*** devananda has joined #openstack-ironic09:01
*** dickson.freenode.net sets mode: +o devananda09:01
*** jistr has quit IRC09:02
*** jistr has joined #openstack-ironic09:02
*** saju_m has quit IRC09:19
*** max_lobur has quit IRC09:22
agordeevmorning Ironic09:26
*** saju_m has joined #openstack-ironic09:34
*** Alexei_987 has joined #openstack-ironic09:43
openstackgerritZhang Yang proposed a change to openstack/ironic: Remove redundant default value None for dict.get  https://review.openstack.org/7777109:58
*** athomas has joined #openstack-ironic10:00
*** romcheg1 has joined #openstack-ironic10:03
*** romcheg has quit IRC10:06
*** max_lobur has joined #openstack-ironic10:11
*** martyntaylor has joined #openstack-ironic10:13
*** romcheg1 is now known as romcheg10:13
shortstopmdurnosvistov, ping10:13
mdurnosvistovshortstop: pong10:14
mdurnosvistovhi10:14
shortstopmdurnosvistov, Hi :-) Thanks for your review on 73054, I don't see decorators being used anywhere else. Should it not come as a separate patch?10:15
shortstopmdurnosvistov, or has it been decided that new patches must make that change?10:15
*** mdurnosvistov has quit IRC10:22
*** mdurnosvistov has joined #openstack-ironic10:29
mdurnosvistovshortstop: There are two patches [1][2], but review it will only of Juno, and I think we should stick decorators. [1] https://review.openstack.org/#/c/73223/ ,[2] https://review.openstack.org/#/c/73256/10:34
*** romcheg has left #openstack-ironic10:36
shortstopmdurnosvistov, I do think that the mock decorators are great. However i think it'll be best to rebase over whichever patch goes in first. Making changes related to a future patch may not be such a good idea IMO!10:39
mdurnosvistovshortstop: rebase it's a good idea! =)10:46
shortstopmdurnosvistov, However, if there's an otherwise compelling reason to make these changes that I can't think of(quite probable :-p), I'd love to hear it! :-)10:49
*** ifarkas has quit IRC11:08
*** ifarkas has joined #openstack-ironic11:10
*** killer_prince2 has quit IRC11:12
*** loki_ has quit IRC11:20
*** killer_prince2 has joined #openstack-ironic11:26
*** killer_prince2 has quit IRC11:31
*** killer_prince2 has joined #openstack-ironic11:37
*** Lord_Set has quit IRC11:38
*** lazy_prince has joined #openstack-ironic11:45
*** russell_h_ has joined #openstack-ironic11:50
*** morgabra_ has joined #openstack-ironic11:51
*** Manishanker has quit IRC11:54
*** killer_prince has quit IRC11:55
*** lazy_prince is now known as killer_prince11:56
*** russell_h has quit IRC11:56
*** russell_h_ is now known as russell_h11:56
*** morgabra has quit IRC11:56
*** bigjools has quit IRC11:56
*** bigjools has joined #openstack-ironic11:56
*** antonym has quit IRC11:56
*** sbadia has quit IRC11:56
*** bigjools has quit IRC11:57
*** bigjools has joined #openstack-ironic11:57
*** bigjools has quit IRC11:57
*** bigjools has joined #openstack-ironic11:57
*** bigjools has quit IRC11:57
*** bigjools has joined #openstack-ironic11:57
*** killer_prince2 has quit IRC11:57
*** Manishanker has joined #openstack-ironic11:57
*** sbadia has joined #openstack-ironic11:58
*** antonym has joined #openstack-ironic11:58
*** killer_prince2 has joined #openstack-ironic12:05
ManishankerHi am trying to install Ironic with devstack using the link  https://etherpad.openstack.org/p/IronicAndDevstackAgain . I am getting this error "[ERROR] /home/test/devstack/functions-common:596 git call failed: [git clone git://git.openstack.org/openstack/requirements.git /opt/stack/requirements]" Can someone help me on this?12:08
*** killer_prince2 has quit IRC12:12
HaomengManishanker: maybe your network is not stable12:13
Haomengcan you try with command by manually12:13
ManishankerYes i have tried it manually12:13
HaomengManishanker: or at that time, git server is not avalible12:13
ManishankerThis is the error fatal: unable to connect to git.openstack.org: git.openstack.org[0: 192.237.223.224]: errno=Connection timed out git.openstack.org[1: 2001:4800:7813:516:3bc3:d7f6:ff04:aacb]: errno=Network is unreachable12:13
HaomengManishanker: so this should be network issue12:14
ManishankerOkay12:14
Haomengcan you ping git.openstack.org12:14
ManishankerYes i can ping12:14
Haomeng64 bytes from git.openstack.org (192.237.223.224): icmp_req=1 ttl=128 time=253 ms12:15
ManishankerPING git.openstack.org (192.237.223.224) 56(84) bytes of data. 64 bytes from git.openstack.org (192.237.223.224): icmp_req=1 ttl=32 time=288 ms12:15
ManishankerI got the same response12:15
HaomengManishanker: I just help you to verify, it works with my env12:16
HaomengManishanker: http://paste.openstack.org/show/72573/12:16
HaomengManishanker: so can you check if you can access the git.openstack.org via http or https12:17
ManishankerYes i can access with https12:17
HaomengManishanker: try with "wget git.openstack.org"12:17
ManishankerYes wget is working12:18
HaomengManishanker: and wget https://git.openstack.org12:19
HaomengManishanker: can you run "git clone git://git.openstack.org/openstack/requirements.git" in your tmp folder, and paste the output to share to me?12:19
ManishankerYes this is also working12:19
HaomengManishanker: network is fine12:19
HaomengManishanker: so we have to check what git error message treturned12:20
ManishankerOkay on it12:20
HaomengManishanker: good luck:)12:20
ManishankerHard luck Its not working .12:22
ManishankerSame error .Network is unreachable12:22
Manishanker:(12:22
ManishankerCloning into 'requirements'... fatal: unable to connect to git.openstack.org: git.openstack.org[0: 192.237.223.224]: errno=Connection timed out git.openstack.org[1: 2001:4800:7813:516:3bc3:d7f6:ff04:aacb]: errno=Network is unreachable12:23
HaomengManishanker: why ipv6 address is used12:23
HaomengManishanker: can you disable ipv6 and try again12:24
HaomengManishanker: or try with ipv4 address - git clone git://192.237.223.224/openstack/requirements.git12:25
HaomengManishanker: should work12:25
Manishanker1 min12:25
HaomengManishanker: k12:25
ManishankerHaomeng : I am using a putty to login to a vm which is in private network .vm is able to ping google.com .Is this setup cause of this network issue ?12:29
ManishankerHaomeng : even this didnt work git clone git://192.237.223.224/openstack/requirements.git12:30
HaomengManishanker: same error message with previous case?12:30
ManishankerYes12:31
ManishankerBut,I can ping 192.237.223.22412:31
HaomengManishanker: but the error indicates we are trying to use ipv6  stack12:31
Haomengerrno=Connection timed out git.openstack.org[1: 2001:4800:7813:516:3bc3:d7f6:ff04:aacb]:12:31
HaomengManishanker: So I think git is trying to use ipv6 as high priority12:33
ManishankerHaomeng : Okay12:33
Haomengsuggest to disable ipv6 stack12:33
HaomengManishanker: it is strange, wget works fine12:33
ManishankerHaomeng : Let me disable ipv6 and get back to you12:33
HaomengManishanker: ok:)12:34
ManishankerI was able to install Havana in devstack on the same machine i had no issues then12:42
ManishankerBut when i am trying to install ironic i am having issues with git clone12:42
HaomengManishanker: this is strange, git is same one:)12:42
ManishankerYeah :)12:43
Manishankerdoes this has anything with the Ironic branch ?12:43
Manishankergit checkout -b ironic ?12:43
HaomengManishanker: works now?12:44
HaomengManishanker: git port is 9418, can you try to check your firewall12:44
ManishankerCloning into 'requirements'... fatal: unable to connect to 192.237.223.224: 192.237.223.224[0: 192.237.223.224]: errno=Connection timed out12:45
ManishankerNot working .I have disabled ipv612:45
ManishankerBut i can ping  test@openstackvm7:/tmp$ ping 192.237.223.224 PING 192.237.223.224 (192.237.223.224) 56(84) bytes of data. 64 bytes from 192.237.223.224: icmp_req=1 ttl=32 time=287 ms12:46
HaomengManishanker: try to ping 9418 port with command "telnet 192.237.223.224 9418"12:46
HaomengManishanker: I think your firewall blocks high 9418 port12:46
ManishankerYes telnet is not working.You are Right12:47
HaomengManishanker: if we get "Escape character is '^]'" that is mean your fw issue12:47
ManishankerCan you please tell me how to fix this ?12:47
*** killer_prince2 has joined #openstack-ironic12:47
HaomengManishanker: sorry for confusing with ipv612:47
HaomengManishanker: depends on your networking accessing mode12:48
agordeevManishanker: i think i know the answer12:48
HaomengManishanker: check the fw and disable12:48
Haomengagordeev: great:)12:48
Haomengagordeev: please:)12:48
ManishankerHaomeng : I think there is no problem with ipv6 and ipv4.From the error i could see first it was trying to use ipv4.Then it was using ipv6 of the git.12:50
agordeevManishanker: you just need to change the GIT_BASE in your localrc. If wget works for you, then just replace 'git://' by 'http://'12:50
agordeeveg.: GIT_BASE=${GIT_BASE:-http://git.openstack.org}12:50
HaomengManishanker: yes12:50
HaomengManishanker: not ipv6 issue, sorry for confusing here:)12:50
Haomengagordeev: yes, we can change the protocol12:50
HaomengManishanker: http/https works in your env12:51
ManishankerHaomeng : I learnt a new thing because of that i learnt how to disable ipv6 :) I should probably thank ypu for that :P12:51
Haomengagordeev: another way is that we can try to disable fw which blocking 941812:51
*** ndipanov has quit IRC12:51
HaomengManishanker: :)12:51
agordeevManishanker: just adding "GIT_BASE=http://git.openstack.org" without the quotes to localrc should resolve the issue12:52
Haomengagordeev: yes agree with you:)12:53
Haomengagordeev: good idea12:54
*** killer_prince2 has quit IRC12:54
agordeevHaomeng: anyway, we don't know if Manishanker is allowed to change his fw settings12:54
Haomengagordeev: yes12:54
Haomengagordeev: some time we can not change fw12:54
*** martyntaylor has quit IRC12:54
Haomengagordeev: because we are not fw owner:)12:55
ManishankerYou guys Rock :) Thanks a lot Haomeng,agordeev12:57
ManishankerWorking now :)12:57
HaomengManishanker: :)12:57
HaomengManishanker: welcome:)12:57
ManishankerBut i am sure i have to upgrade the WSME version to 0.6 then only ir-api would work12:58
HaomengManishanker: :)12:58
HaomengManishanker: devstack script will install wsme v0.612:58
*** martyntaylor has joined #openstack-ironic12:59
Haomengagordeev: one question, do you test with this guide - https://etherpad.openstack.org/p/IronicAndDevstackAgain12:59
Haomengagordeev: my problem is after vm is restart12:59
Haomengagordeev: pxe can not get ip address from neutron dhcp/pxe port13:00
Haomengagordeev: do you have any idea/13:00
Haomengagordeev: thks13:00
agordeevHaomeng: i test it, but i need to test in on purely fresh env13:00
Haomengagordeev: ok no worries13:00
Haomengagordeev: :)13:01
agordeevHaomeng: is tftp server up and listening?13:01
Haomengagordeev: let me check13:01
Haomengagordeev: pxe can not get ip from dhcp I think, this is first step, then request tftp files13:01
agordeevHaomeng: ahh, your vm doesn't get address13:02
Haomengagordeev: yes13:02
agordeevHaomeng: could you check that neutron's dnsmasq hosts/opts files contains your vm's MAC ?13:03
Haomengagordeev: dnsmasq --no-hosts --no-resolv --strict-order --bind-interfaces --interface=tap20a15eb8-a2 --except-interface=lo --pid-file=/opt/stack/data/neutron/dhcp/fe95e452-fd56-4461-9d3d-a7b4cc093833/pid --dhcp-hostsfile=/opt/stack/data/neutron/dhcp/fe95e452-fd56-4461-9d3d-a7b4cc093833/host --dhcp-optsfile=/opt/stack/data/neutron/dhcp/fe95e452-fd56-4461-9d3d-a7b4cc093833/opts --leasefile-ro --dhcp13:04
Haomengthis one /opt/stack/data/neutron/dhcp/fe95e452-fd56-4461-9d3d-a7b4cc093833/host  ?13:04
agordeevHaomeng: right! and opts too13:04
Haomengif it does not contains my vm mac, it should be neutron issue?13:04
Haomengagordeev: I think this dnsmasq dhcp service is started by neutron13:04
Haomengagordeev: i checked, there is no my vm mac13:06
agordeevHaomeng: yes, it was started by neutron. Then Ironic PXE driver tells neutron to update this files with fresh info about new ports. Have it done that?13:06
Haomengagordeev: so if our ironic code issue, or neutron issue13:06
agordeevHaomeng: could you check if you have any traces in ironic-conductor log?13:07
Haomengagordeev: ok13:07
Haomenglet me check13:07
agordeevHaomeng: i meant tracebacks :)13:07
Haomengagordeev: should we check nova ironic driver code to see how ironic driver control neutron to create new dnsmasq to serv the dhcp/pxe request13:09
agordeevHaomeng: no, you shouldn't. Nova driver just triggers ironic API13:12
Haomengagordeev: ok13:13
Haomengagordeev: confusing, why this guide https://etherpad.openstack.org/p/IronicAndDevstackAgain does not include the nova ironic driver patch updating?13:13
Haomengagordeev: should we apply our ironic driver patch?13:13
agordeevHaomeng: of course yes. Nova driver patching is missed. What a shame :(13:14
Haomengagordeev: :)13:15
Haomengagordeev: so it work as expected without nova ironic driver:)13:16
agordeevHaomeng: ha, nova update is placed in localrc13:18
agordeevHaomeng: by >> NOVA_BRANCH=refs/changes/26/71026/1513:19
*** jistr is now known as jistr|english13:19
Haomengagordeev: yes, so that is inclouded already:)13:19
Haomengagordeev: :)13:19
Haomengagordeev: else we can not boot ironic bm13:19
Haomengagordeev: should be this api update_port_dhcp_opts to update neutron port to enable dhcp/pxe13:21
Haomengagordeev: but we did not log message for the neutron updating...13:22
agordeevHaomeng: at first the neutron port should be created (may be that correponds to ironic nova driver). Then updated by conductor PXE driver.13:23
Haomengagordeev: ok, let me check the code and log, thanks:)13:24
Haomengagordeev: nice day:)13:24
agordeevHaomeng: have a nice day too :)13:27
Haomengagordeev: :)13:28
*** linggao has joined #openstack-ironic13:32
agordeevHaomeng: i've just checked the logs. Nova-compute call neutronclient to create the port.13:32
Haomengagordeev: ok, thanks for your confirmation13:33
*** nosnos_ has quit IRC13:45
*** krtaylor has quit IRC13:46
*** nosnos has joined #openstack-ironic13:48
*** nosnos has quit IRC13:49
*** nosnos has joined #openstack-ironic13:50
*** nosnos has quit IRC13:54
*** dhellmann_ is now known as dhellmann14:14
*** saju_m has quit IRC14:15
*** ndipanov has joined #openstack-ironic14:23
*** ndipanov has quit IRC14:24
*** ndipanov has joined #openstack-ironic14:24
*** jistr|english is now known as jistr14:25
devanandagood morning14:35
devanandaagordeev: Haomeng: it looks like you two are discussing the devstack support, and have figured it all out :)14:36
*** krtaylor has joined #openstack-ironic14:40
*** jgrimm has quit IRC14:52
*** Manishanker has quit IRC14:52
openstackgerritDevananda van der Veen proposed a change to openstack/ironic: Import Nova "ironic" driver  https://review.openstack.org/7800215:01
*** russell_h has quit IRC15:03
*** russell_h has joined #openstack-ironic15:03
agordeevdevananda: good morning15:11
NobodyCamgood morning15:15
NobodyCamgah running a bit late15:15
NobodyCamlazy morning15:15
russell_hyou've got a different definition of "lazy morning" than I do, given that its 7:15am15:16
NobodyCamlol normaly up by 615:16
NobodyCamlol15:16
NobodyCamdevananda: haha I was working on generate_sameple last night15:17
NobodyCamlol I was thinking along these lines: http://paste.openstack.org/show/wPJ9S6r8Ku2sB9k1v1ul/15:18
*** martyntaylor has quit IRC15:19
agordeevNobodyCam: g'morning15:20
NobodyCamgood mornig agordeev :)15:20
max_loburmorning Ironic %)15:20
NobodyCammorning max_lobur15:21
*** martyntaylor has joined #openstack-ironic15:21
GheRiveromorning @all!15:22
agordeevmax_lobur: morning :)15:23
agordeevGheRivero: morning :)15:23
*** dwalleck has joined #openstack-ironic15:23
linggaogood morning.15:24
*** jbjohnso has joined #openstack-ironic15:27
*** coolsvap has joined #openstack-ironic15:27
NobodyCammorning GheRivero morning linggao15:28
linggao:)15:28
yuriyzmorning/evening all15:42
NobodyCammorning / evening yuriyz15:50
*** rloo has joined #openstack-ironic15:54
*** jgrimm has joined #openstack-ironic15:55
*** ifarkas has quit IRC16:05
*** Alexei_987 has quit IRC16:10
*** vkozhukalov has joined #openstack-ironic16:21
*** lsmola has quit IRC16:41
*** vkozhukalov has quit IRC16:42
*** rloo has quit IRC16:44
devanandag'morning (again) :)16:48
*** rloo has joined #openstack-ironic16:54
*** yuriyz has quit IRC16:59
*** jgrimm has quit IRC17:00
*** rwsu has joined #openstack-ironic17:03
ShrewsNobodyCam, devananda: you are not here. why aren't you here? you should be here. get here.17:07
*** harlowja has joined #openstack-ironic17:07
*** ifarkas has joined #openstack-ironic17:07
*** jdob has joined #openstack-ironic17:08
*** matty_dubs|gone is now known as matty_dubs|sfo17:11
*** jcooley_ has joined #openstack-ironic17:12
*** derekh has joined #openstack-ironic17:14
NobodyCamI'm here17:15
NobodyCamI was getting coffee17:15
*** dwalleck has quit IRC17:20
*** krtaylor has quit IRC17:23
*** dwalleck has joined #openstack-ironic17:26
*** dwalleck_ has joined #openstack-ironic17:28
*** jcooley_ has quit IRC17:28
*** ndipanov has quit IRC17:28
JoshNang_good morning!17:30
*** krtaylor has joined #openstack-ironic17:30
*** dwalleck has quit IRC17:31
*** JoshNang_ is now known as JoshNang17:32
*** openstackgerrit has quit IRC17:32
*** vkozhukalov has joined #openstack-ironic17:34
*** lucasagomes has joined #openstack-ironic17:34
devanandaShrews: i'm here. where are you?17:34
devanandaagordeev: ping17:34
lucasagomesdevananda, ipmitool mc guid?17:36
NobodyCammorning JoshNang :)17:37
*** blamar has joined #openstack-ironic17:39
*** max_lobur has quit IRC17:40
*** athomas has quit IRC17:42
jrollmornin y'all17:42
*** martyntaylor has quit IRC17:56
lucasagomesdevananda, busy?18:01
lucasagomesdevananda, can we chat a bit with matty_dubs|sfo re ironic managing storage arrays18:02
devanandalucasagomes: sure18:03
lucasagomesack, there's no seats back here so it's better to go to matty_dubs|sfo18:03
lucasagomesNobodyCam, u too18:05
lucasagomesrloo, u too18:05
*** athomas has joined #openstack-ironic18:05
jrollhow do we currently handle migrations?18:09
jrolldatabase migrations*18:18
jrollare there docs on that or just figure out alembic and do that?18:18
*** openstackgerrit has joined #openstack-ironic18:26
*** dwalleck has joined #openstack-ironic18:26
*** jcooley_ has joined #openstack-ironic18:27
*** dwalleck_ has quit IRC18:30
*** jistr has quit IRC18:35
*** martyntaylor has joined #openstack-ironic18:52
*** early has quit IRC18:58
*** jcooley_ has quit IRC18:59
*** jcooley_ has joined #openstack-ironic18:59
*** openstackgerrit has quit IRC19:02
*** jcooley_ has quit IRC19:04
*** early has joined #openstack-ironic19:04
*** jcooley_ has joined #openstack-ironic19:04
*** jcooley_ has quit IRC19:05
*** traiden has joined #openstack-ironic19:14
*** epim has joined #openstack-ironic19:16
linggaodevananda, are there any docs for setting up a cloud with tripleo/devstack and ironic? I'd like to setup a cloud with real hardware to test ironic node deployments.19:18
jrolllinggao: this worked fine for me: https://wiki.openstack.org/wiki/Ironic/Devstack19:19
linggaojroll, thanks, exacly what I was looking for. Does it install the ironic baremetal driver into nova?19:20
devanandalinggao: no, we're going to start writing them soon19:20
Shrewsdevananda: is the devstack patch mucking up your dns resolution on your hp cloud instance too?19:21
devanandaShrews: yes. 8.8.8.8 is your friend19:21
jrolllinggao: no, just gets ironic running. for my testing I've been curling the ironic api19:21
devanandajroll: it should be installing python-ironicclient19:21
devanandajroll: but you can use curl if you want to19:22
linggaooh, I'd like to see nova boot command working for ironic.19:22
devanandalinggao: "nova boot" depends also on the driver in Nova, which has not landed in trunk. we're workign on simplifying that19:22
linggaodevananda, I see.19:23
*** coolsvap has quit IRC19:24
linggaodevananda, can Ironic graduate from incubation in icehouse time frame?19:25
jrollno :(19:26
linggaojroll, for real?19:27
jrollfor real.19:27
*** dwalleck has quit IRC19:27
jrollit's already been decided19:28
linggaojroll, I see. I guess there is too much work to do in ironic to get it graduated.19:28
devanandathe TC will still need to vote, which hasn't happened yet, but there's enough concensus that I do not expect that Ironic will graduate19:28
devanandait's not the end of the world :)19:29
devanandaand doesn't change anything as far as our development roadmap19:29
jroll^19:29
jrollcode that hasn't graduated can still be executed :)19:30
linggaodevananda, true. we'll shoot for Juno.19:30
devanandawe will still have an Icehouse release, and we should make it as good as possible19:30
adam_gare the libvirt version/feature requirements for ironic documented somewhere? running into issues /w newer libvirt on rackspace cloud, which will cause issues if devstack+ironic is meant to be run there as part of the gate19:32
*** jcooley_ has joined #openstack-ironic19:33
comstudthere should not be a libvirt version dependency19:33
adam_ghttps://bugs.launchpad.net/ubuntu/+bug/124802519:33
comstudi can't imagine why?19:33
comstudbecause you don't use libvirt for ironic19:33
adam_gsorry--meant requirements for ironic+devstack19:34
adam_gas per https://etherpad.openstack.org/p/IronicAndDevstackAgain19:35
comstudah, devstack19:35
matty_dubs|sfolifeless: http://fedoraproject.org/wiki/Infrastructure/Mirroring#Mirroring19:36
matty_dubs|sfoDescription path on master server Comments19:38
devanandacomstud: there's totally a libvirt requriement19:42
devanandacomstud: because integration testing for ironic is going to create some VMs out of band to mock hardware nodes19:43
*** jcooley_ has quit IRC19:48
comstudyeah,i get it, for CI19:48
*** jcooley_ has joined #openstack-ironic19:48
comstudthe question was generic enough to begin with that I didn't connect it to gating19:49
comstudwell19:50
comstudactually, I just can't read. that's all. :)19:50
*** dwalleck has joined #openstack-ironic19:51
*** vkozhukalov has quit IRC19:53
devanandahehe19:58
*** morgabra_ is now known as morgabra20:05
*** morgabra has joined #openstack-ironic20:05
*** traiden has quit IRC20:06
*** reaper has joined #openstack-ironic20:28
*** jcooley_ has quit IRC20:32
*** epim has quit IRC20:38
*** martyntaylor has quit IRC20:51
jrollhow do we currently handle migrations? are there ironic docs for this, or should I just figure out alembic, generate the migration, and that's it?20:54
devanandajroll: yes20:54
jrollyes, what?20:55
devanandajroll: what are you migrating?20:55
jrolldevananda: adding that instance_info field20:55
devanandajroll: alembic20:55
jrollok, thanks :)20:56
jrolljust wanted to be sure there's nothing else to do20:56
devanandajroll: should have db migration test too20:56
*** mrda_away is now known as mrda20:56
mrdamorning!20:57
devanandamrda: hi there!20:57
jrolldevananda: ok, that's the sort of thing I was looking for :)20:57
jrollheya mrda20:57
mrdasprint going well?20:58
jrolloh yeah20:58
mrdagood to hear20:58
*** vkozhukalov has joined #openstack-ironic20:59
*** martyntaylor has joined #openstack-ironic21:04
*** athomas has left #openstack-ironic21:04
*** dwalleck_ has joined #openstack-ironic21:13
*** dwalleck has quit IRC21:16
*** dwalleck has joined #openstack-ironic21:17
*** dwalleck_ has quit IRC21:21
*** dwalleck has quit IRC21:36
*** ifarkas has quit IRC21:40
*** dwalleck has joined #openstack-ironic21:46
*** linggao has quit IRC21:47
*** imtiaz has joined #openstack-ironic21:49
jrollimtiaz: https://wiki.openstack.org/wiki/Ironic/Devstack21:49
*** max_lobur has joined #openstack-ironic21:51
*** vkozhukalov has quit IRC21:54
JayFrussell_h: https://blueprints.launchpad.net/ironic/+spec/utility-ramdisk  RFR on the updated description? It should somewhat describe what teeth-agent does/will do22:06
*** dwalleck_ has joined #openstack-ironic22:08
russell_hJayF: is it already updated? It more or less LGTM22:08
JayFyeah I just updated it :)22:08
russell_hI'm not an expert on, for example, iSCSI22:08
russell_hbut is that really "hardware configuration?22:08
max_loburhi devananda, I see you're rebasing my patch 74063. are there some problems (except rebase)? I can fix now if needed22:09
*** dwalleck has quit IRC22:11
GheRiverolucasagomes: ping. Can you do a quick high view review on https://review.openstack.org/#/c/78384/ (Generic MixedVendorInterface using static mapping)22:12
lucasagomesGheRivero, hey yeah, just finishing something up here and will take a look22:13
GheRiverolucasagomes: thx!22:13
*** dwalleck has joined #openstack-ironic22:14
max_loburHi GheRivero22:14
max_loburyou decided to implement alternate approach as a separate patch?22:15
GheRiverowe decided to et rid of any dynamic mapping, so previous patches were abandoned22:16
max_loburah, ok22:16
rloomax_lobur: wrt patch 74063. I think deva fixed the issue I mentioned (see inline comment.)22:16
max_loburhttps://review.openstack.org/#/c/74063/5/ironic/conductor/manager.py22:17
max_loburvalidate?22:17
*** dwalleck_ has quit IRC22:17
max_loburI guess that should be fixed in drivers22:17
rloommax_lobur. yeah, validate. i opened a bug to fix in drivers.22:17
max_loburto make all them not return True/False22:18
rloomax_lobur. yup.22:18
max_loburis there something that should be changed in my patch22:18
max_loburI did not check those return value22:18
max_loburso I assume it should be fine as is22:18
rloomax_lobur. you had something like driver_data = ... validate.22:20
max_loburlemme see22:20
max_loburahh you're right22:21
max_loburyea I see he fixed this22:21
max_loburok, thanks!22:21
rloomax_lobur. i just looked at deva's change. it isn't complete. maybe my fault. i didn't point out everywhere it needed to change.22:22
rloomax_lobur. rpcapi.py also needs to change.22:22
rloomax_lobur. should i -1 or are you OK with making the changes?22:23
max_loburno I think it's standard for rpc api22:23
max_loburwe even have test for this22:23
max_loburit will just return None as call method22:24
max_loburI think it's OK22:24
max_loburwe have a generic test for all rpcapi methods, if CALL is used, it checks that return value is pushed up from CALL22:24
*** vkozhukalov has joined #openstack-ironic22:25
rloomax_lobur. but the docstring says" returns: driver data returned from driver.validate...22:25
*** dwalleck has quit IRC22:25
max_loburhttps://github.com/openstack/ironic/blob/master/ironic/tests/conductor/test_rpcapi.py#L10322:25
max_loburhmm, actually both doc strings22:26
max_loburyes, I'll quickly fix this now22:26
rloomax_lobur. thx!22:27
max_loburthank you for this findings22:28
rloomax_lobur. yw :-)22:32
max_loburgerrit is not posting messages about new patches anymore? :)22:33
*** dwalleck has joined #openstack-ironic22:39
*** jbjohnso has quit IRC22:40
*** dwalleck_ has joined #openstack-ironic22:40
*** dwalleck has quit IRC22:43
max_loburGheRivero: new multiplexed vendor looks cool22:46
rloomax_lobur? I got email about your revision?22:47
max_loburI mean previously it posted to this chat too22:48
*** jcooley_ has joined #openstack-ironic22:49
GheRiveromax_lobur: cool22:50
*** vkozhukalov has quit IRC22:55
*** vkozhukalov has joined #openstack-ironic22:56
*** openstack has joined #openstack-ironic23:03
*** dwalleck_ has quit IRC23:06
*** dwalleck_ has joined #openstack-ironic23:06
devanandamax_lobur: 74063 needs a bug ref23:09
max_loburhi deva, did we have bug for this?23:10
max_loburprevious patch fixed "race condition for change power state"23:10
max_loburwe did not have common bug for async workers AFAIR23:11
max_loburwe just decided to expand this approach to other fields23:11
max_loburshould I fill one for this?23:11
rloomax_lobur. yeah, I think a bug ought to be open for this. it has to do with code freeze etc.23:18
max_loburk, opening23:19
rloomax_lobur. deva sent email about what we should be focusing on, etc.etc (might be my fault about the bug, i was asking him questions...)23:20
rloomax_lobur. eg, i wanted to know if i should be reviewing things that had no bugs associated with them.23:21
*** salseo69 has joined #openstack-ironic23:21
*** imtiaz has quit IRC23:23
max_loburk, lets wait for him. Basically there was a bug "# NOTE(deva): There is a race condition in the RPC API for vendor_passthru." I guess we did not have it filled on launchpad23:24
*** imtiaz has joined #openstack-ironic23:24
*** salseo69 has left #openstack-ironic23:24
*** dwalleck_ has quit IRC23:25
devanandao/23:26
max_loburso, to fill or not to fill23:27
*** imtiaz has quit IRC23:28
*** vkozhukalov has quit IRC23:29
max_loburUnicodeDecodeError: 'ascii' codec can't decode byte 0xe2 in position 72: ordinal not in range(128)23:33
max_loburanyone seen this in tox?23:33
max_loburwanted to rebuild my env and got this23:34
NobodyCamhumm23:34
NobodyCamI have not seen that23:34
jrollmax_lobur: got a full traceback?23:34
max_loburafter Running cffi-0.8.1/setup.py -q bdist_egg --dist-dir /tmp/easy_install-WTiVvi/cffi-0.8.1/egg-dist-tmp-WFeb_k23:35
max_loburyeh23:35
max_loburlemme paste23:35
devanandalucasagomes: reviewed 7804723:35
lucasagomesdevananda, ack thanks23:36
lucasagomesdevananda, I put some info on the patch importing the nova driver in ironic23:36
max_loburhere it is http://paste.openstack.org/show/72730/23:36
devanandalucasagomes: did we have GET support for vendor+_passthru?23:37
devanandaif so, we need to leave the "return $soemthign" in 74063 after all23:37
*** Slower has joined #openstack-ironic23:37
jrollmax_lobur: you need libffi-dev23:37
devanandait should not be "driver_info = driver.validate && return driver_info"23:37
devanandasmoethingmore like this:23:38
Slowerlove the name of this project.. I'm still chuckling about it23:38
Slowerwhoever thought it up is a genius in my books :)23:38
lucasagomesdevananda, not in the moment but it won't be hard to add23:38
NobodyCamSlower: :-p23:38
devanandalucasagomes: ok -- if we dont support GET now, then that's fine, but it will be needed23:38
lucasagomesu just need to add a "def get_one" method in our NodeVendorPassthruController23:38
lucasagomesdevananda, ack23:38
lucasagomesI can take a look at it23:39
max_loburk, so leave 74063 as is?23:39
lucasagomesdevananda, btw, about validate raising an exception23:39
lucasagomeswe want that to bubble up to the api no?23:39
lucasagomesthat's why I didn't wrapped it in a try...except block23:39
max_loburjroll: thanks!23:39
devanandaooh23:39
devanandai see23:39
max_loburinstalled & trying again23:40
jrollmax_lobur: np. I ran into this earlier. going to add it to the setup guide or whatever23:40
max_loburfor exceptions - we're doing reraise there23:40
max_loburjroll: yup, thanks!23:40
jrollmax_lobur: https://review.openstack.org/#/c/78454/23:43
jroll^ if someone wants to verify that the fedora package I added is the correct package, that would be great.23:44
max_loburI don't have fedora currently :/23:44
max_loburI'll +1 once my tox finished23:45
jrollno worries, I know someone around here does23:45
jrollI forget who23:45
max_loburlucas :)23:45
max_loburlucasagomes: https://review.openstack.org/#/c/78454/ pls take a quick look23:46
JayFjroll: https://apps.fedoraproject.org/packages/libffi-devel23:46
JayFjroll: appeared to be when I checked23:46
max_loburcool23:46
jrollJayF: yeah, I found that. I'd like confirmation that an environment rebuild works :P23:46
max_loburjroll: py26 passed, so far so good :)23:59
jroll:)23:59
jrollmax_lobur: I just had this happen this morning so I'm pretty confident about it :P23:59

Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!