[Date Prev][Date Next]   [Thread Prev][Thread Next]   [Thread Index] [Date Index] [Author Index]

RE: a weird problem: app create fails, but app is really created and accessible



Hi Luke, 

Which config file specifies the ProxyTimeout option?

Thanks,
Shoumeng


-----Original Message-----
From: Luke Meyer [mailto:lmeyer redhat com] 
Sent: Monday, June 10, 2013 9:32 AM
To: Yan, Shoumeng
Cc: dev lists openshift redhat com
Subject: Re: a weird problem: app create fails, but app is really created and accessible

A 502 error is basically the proxy reporting that something went wrong with the response (i.e. it didn't get one).

On the broker host there is an httpd proxy externally-facing on the host which proxies back to the broker webapp (which binds to localhost). Here it looks like the app create took about 70s. Perhaps the ProxyTimeout on the host httpd is set to 60 or some such? It really should be set to 240 or higher; the broker is often sitting around waiting for the node to do something. The only real error is that the success of the operation didn't make it back to the client.

----- Original Message -----
From: "Shoumeng Yan" <shoumeng yan intel com>
To: dev lists openshift redhat com
Sent: Saturday, June 8, 2013 11:27:43 PM
Subject: a weird problem: app create fails, but app is really created and	accessible





Hi there, 



My openshift origin installation met a weird issue. When I create app, it reports some error (code 502). But the app is actually created (rhc apps show the app) and also accessible using git and ssh. 



I attached below my related logs where I did not see an obvious issue. Hope someone could provide some hints on further investigation. Thanks. 



Thanks, 

Shoumeng 



[root localhost ~]# rhc app create mydiy7 diy -d 

DEBUG: Connecting to https://localhost.localdomain/broker/rest/api 

DEBUG: Getting all cartridges 

DEBUG: Request GET https://localhost.localdomain/broker/rest/api 

DEBUG: SSL Verification failed -- Using self signed cert 

DEBUG: code 200 1382 ms 

DEBUG: Request GET https://localhost.localdomain/broker/rest/cartridges 

DEBUG: SSL Verification failed -- Using self signed cert 

DEBUG: code 200 704 ms 

Using diy-0.1 (Do-It-Yourself) for 'diy' 

DEBUG: Getting all domains 

DEBUG: Request GET https://localhost.localdomain/broker/rest/domains 

DEBUG: SSL Verification failed -- Using self signed cert 

DEBUG: code 200 793 ms 

DEBUG: Getting all domains 

DEBUG: Finding domain mypaas 

DEBUG: Getting all domains 



Application Options 

------------------- 

DEBUG: Getting all domains 

Namespace: mypaas 

Cartridges: diy-0.1 

Gear Size: default 

Scaling: no 



Creating application 'mydiy7' ... DEBUG: Creating application 'mydiy7' with these options - {:cartridges=>["diy-0.1"]} 

DEBUG: Adding application mydiy7 to domain mypaas 

DEBUG: Request POST https://localhost.localdomain/broker/rest/domains/mypaas/applications 

DEBUG: SSL Verification failed -- Using self signed cert 

DEBUG: code 502 60483 ms 

DEBUG: ERROR: Received bad gateway from server, will retry once if this is a GET 

An error occurred while communicating with the server. This problem may only be 

temporary. Check that you have correctly specified your proxy server 

'http://proxy-sc.mycorp.com:1111' as well as your OpenShift server 

'https://localhost.localdomain/broker/rest/domains/mypaas/applications'. 



[root localhost broker]# tail /var/log/openshift/broker/user_action.log 

SUCCESS DATE=2013-06-09 TIME=11:06:15 ACTION=LIST_CARTRIDGES REQ_ID=6659e6c04d9a1ac51c5d08789c8a21ac USER_ID= LOGIN= List all cartridges 

SUCCESS DATE=2013-06-09 TIME=11:06:16 ACTION=AUTHENTICATE REQ_ID=7b06fe00205d9854c9980e3ab50e6fe0 USER_ID=51ae13cd30050a3592000001 LOGIN=admin IP=127.0.0.1 SCOPES=session Authenticated 

SUCCESS DATE=2013-06-09 TIME=11:06:17 ACTION=LIST_DOMAINS REQ_ID=7b06fe00205d9854c9980e3ab50e6fe0 USER_ID=51ae13cd30050a3592000001 LOGIN=admin 

SUCCESS DATE=2013-06-09 TIME=11:06:17 ACTION=AUTHENTICATE REQ_ID=90d9ed2251371ddc6755604de61019bf USER_ID=51ae13cd30050a3592000001 LOGIN=admin IP=127.0.0.1 SCOPES=session Authenticated 

SUCCESS DATE=2013-06-09 TIME=11:07:27 ACTION=ADD_APPLICATION REQ_ID=90d9ed2251371ddc6755604de61019bf USER_ID=51ae13cd30050a3592000001 LOGIN=admin APP=mydiy7 DOMAIN=mypaas APP_UUID=51b3f129e13823d5d300002c Application mydiy7 was created. 



[root localhost broker]# tail /var/log/openshift/broker/development.log 

MOPED: 127.0.0.1:27017 UPDATE database=openshift_broker_dev collection=applications selector={"_id"=>"51b3f129e13823d5d300002c"} update={"$set"=>{"pending_op_groups.0.pending_ops.11.state"=>:completed}} flags=[] 

COMMAND database=openshift_broker_dev command={:getlasterror=>1, :safe=>true} (2.5373ms) 

MOPED: 127.0.0.1:27017 UPDATE database=openshift_broker_dev collection=applications selector={"_id"=>"51b3f129e13823d5d300002c"} update={"$pull"=>{"pending_op_groups"=>{"_id"=>"51b3f129e13823d5d300002d"}}} flags=[] 

COMMAND database=openshift_broker_dev command={:getlasterror=>1, :safe=>true} (4.5893ms) 

MOPED: 127.0.0.1:27017 QUERY database=openshift_broker_dev collection=applications selector={:_id=>"51b3f129e13823d5d300002c"} flags=[] limit=-1 skip=0 batch_size=nil fields=nil (0.2468ms) 

MOPED: 127.0.0.1:27017 QUERY database=openshift_broker_dev collection=domains selector={"$query"=>{"_id"=>"51ae13da30050a3592000005"}, "$orderby"=>{:_id=>1}} flags=[] limit=-1 skip=0 batch_size=nil fields=nil (8.6658ms) 

MOPED: 127.0.0.1:27017 COMMAND database=openshift_broker_dev command={:findAndModify=>"locks", :query=>{"user_id"=>"51ae13cd30050a3592000001", "locked"=>false, "app_ids.51b3f129e13823d5d300002c"=>{"$exists"=>true}}, :new=>true, :update=>{"$unset"=>{"app_ids.51b3f129e13823d5d300002c"=>""}}} (0.4041ms) 

MOPED: 127.0.0.1:27017 QUERY database=openshift_broker_dev collection=domains selector={:_id=>"51ae13da30050a3592000005"} flags=[] limit=-1 skip=0 batch_size=nil fields=nil (2.2936ms) 

SUCCESS ACTION=ADD_APPLICATION USER_ID=51ae13cd30050a3592000001 LOGIN=admin APP=mydiy7 DOMAIN=mypaas APP_UUID=51b3f129e13823d5d300002c Application mydiy7 was created. Application mydiy7 was created., 172.16.117.49, Disclaimer: This is an experimental cartridge that provides a way to try unsupported languages, frameworks, and middleware on Openshift. 

Completed 201 Created in 70137ms (Views: 17.0ms) 



[root localhost broker]# tail /var/log/openshift/broker/httpd/access_log 

127.0.0.1 localhost.localdomain - - [09/Jun/2013:11:06:13 +0800] "GET /broker/rest/api HTTP/1.1" 200 3450 "-" "rhc/1.7.5 (ruby 1.9.3; x86_64-linux) (2.3.3, ruby 1.9.3 (2012-11-10))" 

127.0.0.1 localhost.localdomain - - [09/Jun/2013:11:06:14 +0800] "GET /broker/rest/cartridges HTTP/1.1" 200 11414 "-" "rhc/1.7.5 (ruby 1.9.3; x86_64-linux) (2.3.3, ruby 1.9.3 (2012-11-10))" 

127.0.0.1 localhost.localdomain - - [09/Jun/2013:11:06:16 +0800] "GET /broker/rest/domains HTTP/1.1" 200 2098 "-" "rhc/1.7.5 (ruby 1.9.3; x86_64-linux) (2.3.3, ruby 1.9.3 (2012-11-10))" 

127.0.0.1 localhost.localdomain - - [09/Jun/2013:11:06:17 +0800] "POST /broker/rest/domains/mypaas/applications HTTP/1.1" 201 6858 "-" "rhc/1.7.5 (ruby 1.9.3; x86_64-linux) (2.3.3, ruby 1.9.3 (2012-11-10))" 



[root localhost broker]# tail /var/log/openshift/node/platform.log 

D, [2013-06-09T11:07:20.482391 #21265] DEBUG -- : Shell command '/usr/sbin/httxt2dbm -f DB -i /etc/httpd/conf.d/openshift/nodes.txt -o /etc/httpd/conf.d/openshift/nodes.db-20130609-21265-9s7xba/new.db' ran. rc=0 out= 

D, [2013-06-09T11:07:20.482580 #21265] DEBUG -- : httxt2dbm: /etc/httpd/conf.d/openshift/nodes: 0: stdout: stderr: 

I, [2013-06-09T11:07:20.803428 #2123] INFO -- : (0) 

------ 

Initialized empty Git repository in /var/lib/openshift/51b3f129e13823d5d300002c/git/mydiy7.git/ 

CLIENT_RESULT: Disclaimer: This is an experimental cartridge that provides a way to try unsupported languages, frameworks, and middleware on Openshift. 



------) 

D, [2013-06-09T11:07:21.426829 #2123] DEBUG -- : Creating v1 model for 51b3f129e13823d5d300002c: initialize 

D, [2013-06-09T11:07:24.715672 #2123] DEBUG -- : Creating v1 model for 51b3f129e13823d5d300002c: initialize 



[root localhost broker]# tail /var/log/openshift/node/platform-trace.log 

D, [2013-06-09T10:56:03.485480 #15611] DEBUG -- : oo_spawn running /usr/sbin/httxt2dbm -f DB -i /etc/httpd/conf.d/openshift/nodes.txt -o /etc/httpd/conf.d/openshift/nodes.db-20130609-15611-1ff1c1i/new.db: {:unsetenv_others=>false, :close_others=>true, :in=>"/dev/null", :out=>#<IO:fd 10>, :err=>#<IO:fd 8>} 

D, [2013-06-09T11:07:19.795604 #21265] DEBUG -- : oo_spawn running service openshift-node-web-proxy reload: {:unsetenv_others=>false, :close_others=>true, :in=>"/dev/null", :out=>#<IO:fd 10>, :err=>#<IO:fd 8>} 

D, [2013-06-09T11:07:19.983377 #21265] DEBUG -- : oo_spawn buffer(9/) Reloading node-web-proxy: 

D, [2013-06-09T11:07:19.997423 #21265] DEBUG -- : oo_spawn buffer(9/) [ 

D, [2013-06-09T11:07:19.997565 #21265] DEBUG -- : oo_spawn buffer(9/) OK 

D, [2013-06-09T11:07:19.997671 #21265] DEBUG -- : oo_spawn buffer(9/) ] 

D, [2013-06-09T11:07:20.008338 #21265] DEBUG -- : oo_spawn buffer(9/) 

D, [2013-06-09T11:07:20.008596 #21265] DEBUG -- : oo_spawn buffer(9/) 



D, [2013-06-09T11:07:20.106700 #21265] DEBUG -- : oo_spawn running /usr/sbin/httxt2dbm -f DB -i /etc/httpd/conf.d/openshift/nodes.txt -o /etc/httpd/conf.d/openshift/nodes.db-20130609-21265-9s7xba/new.db: {:unsetenv_others=>false, :close_others=>true, :in=>"/dev/null", :out=>#<IO:fd 10>, :err=>#<IO:fd 8>} _______________________________________________
dev mailing list
dev lists openshift redhat com
http://lists.openshift.redhat.com/openshiftmm/listinfo/dev


[Date Prev][Date Next]   [Thread Prev][Thread Next]   [Thread Index] [Date Index] [Author Index]