[Date Prev][Date Next]   [Thread Prev][Thread Next]   [Thread Index] [Date Index] [Author Index]

Re: HAProxy gear logs



Thanks Mrunal, Mike!

Well, at my case I just create simple php app (only index.html and couple img on it) with -s and then try to load it with two simultaneously ab. 
At the beginning lab had figure out my load and add additional gear, but after some moments something goes wrong and all scaling feature (up and down) nod working at all. 

What I can see:

From haproxy-status page:

stats
QueueSession rate SessionsBytesDeniedErrors WarningsServer
CurMaxLimit CurMaxLimitCur MaxLimitTotalLbTot InOutReqResp ReqConnRespRetr RedisStatusLastChkWght ActBckChkDwn DwntmeThrtle
Frontend 1 2 -1 13000 45 42977 5795870 00 OPEN
Backend 00 00 003000 0 042977 579587 00 00 00 18m20s UP 0 00  0 

express
QueueSession rateSessions BytesDeniedErrors WarningsServer
CurMaxLimitCur MaxLimitCurMax LimitTotalLbTotIn OutReqRespReq ConnRespRetrRedis StatusLastChkWghtAct BckChkDwnDwntme Thrtle
Frontend 0 16 -6767 3000 95 74541403 903578060 00 0 OPEN
filler 00 -00 00 -0 00 00 0 0 00 1- Y-
gear-27b9d03820-nspace 00 -122131 29 30- 117210 11720537267033 451714235 0 0 0 0018m20s UP L7OK/200 in 249ms 1Y- 0 00s-
gear-cb5ac386ba-nspace 00 -122131 29 30- 117244 11724437274370 451863825 0 0 0 0018m20s UP L7OK/200 in 228ms 1Y- 0 00s-
gear-d7f86b8193-nspace 00 -04 03 -12 40 00 0 0 8418m19s DOWN L4CON in 0ms 1Y -0 118m19s -
local-gear 00 -00 00 -0 00 00 0 0 00 0Y --
Backend 0 0 244261 58 603000 234454 234453 74541403 903578060 0 0 0 0 84 18m20s UP  22 1  00s

From haproxy-1.4/logs/scale_events.log for that app:

# Logfile created on Sat Jan 12 19:35:15 +0200 2013 by logger.rb/1.2.6
I, [2013-01-12T19:35:21.708074 #17464]  INFO -- : Starting haproxy_ctld
D, [2013-01-12T19:35:21.711129 #17464] DEBUG -- : GEAR_INFO - capacity: 0.0% gear_count: 1 sessions: 0 up/remove_thresh: 90.0%/49.9% sec_left_til_remove: 120 gear_remove_thresh: 0/20
I, [2013-01-12T19:43:00.607716 #21946]  INFO -- : Starting haproxy_ctld
D, [2013-01-12T19:43:00.610906 #21946] DEBUG -- : GEAR_INFO - capacity: 0.0% gear_count: 1 sessions: 0 up/remove_thresh: 90.0%/49.9% sec_left_til_remove: 120 gear_remove_thresh: 0/20
I, [2013-01-12T19:45:40.098578 #21946]  INFO -- : GEAR_UP - capacity: 90.0% gear_count: 1 sessions: 9 up_thresh: 90.0%
I, [2013-01-12T19:46:44.906404 #23436]  INFO -- : Starting haproxy_ctld
D, [2013-01-12T19:46:44.920943 #23436] DEBUG -- : GEAR_INFO - capacity: 40.0% gear_count: 1 sessions: 4 up/remove_thresh: 90.0%/49.9% sec_left_til_remove: 120 gear_remove_thresh: 0/20
I, [2013-01-12T19:46:45.006991 #23436]  INFO -- : GEAR_UP - capacity: 110.0% gear_count: 1 sessions: 11 up_thresh: 90.0%
I, [2013-01-12T19:47:09.326502 #23662]  INFO -- : Starting haproxy_ctld
D, [2013-01-12T19:47:09.447347 #23662] DEBUG -- : GEAR_INFO - capacity: 50.0% gear_count: 2 sessions: 10 up/remove_thresh: 90.0%/49.9% sec_left_til_remove: 120 gear_remove_thresh: 0/20
I, [2013-01-12T19:47:45.916311 #24023]  INFO -- : Starting haproxy_ctld
D, [2013-01-12T19:47:46.014610 #24023] DEBUG -- : GEAR_INFO - capacity: 50.0% gear_count: 2 sessions: 11 up/remove_thresh: 90.0%/49.9% sec_left_til_remove: 120 gear_remove_thresh: 0/20
I, [2013-01-12T19:48:01.015038 #24165]  INFO -- : Starting haproxy_ctld
D, [2013-01-12T19:48:01.019328 #24165] DEBUG -- : GEAR_INFO - capacity: 30.0% gear_count: 3 sessions: 11 up/remove_thresh: 90.0%/49.9% sec_left_til_remove: 120 gear_remove_thresh: 0/20

Last two messages at /var/log/mcollective.log at that node where haproxy gear for this app reside:

I, [2013-01-12T19:48:01.145103 #1410]  INFO -- : openshift.rb:612:in `reap_output' cartridge_do_action (0)
------
Sat Jan 12 19:47:48 EET 2013: Conditionally reloading HAProxy service

------)
I, [2013-01-12T19:48:01.147980 #1410]  INFO -- : openshift.rb:572:in `execute_parallel_action' execute_parallel_action call - 10 taggear91153799a9db4afe8babf9a37559a619jobargs--gear-uuid91153799a9db4afe8babf9a37559a619--input-argsm4 nspace 91153799a9db4afe8babf9a37559a619 \'27b9d03820f4451085fa0aa470062fcc\'\=\'27b9d03820-nspace.oshift.labspace.studiogrizzly.com\|192.168.2.151:35541'
'\'\ \'cb5ac386ba0343e6b99ae3d12095db63\'\=\'cb5ac386ba-nspace.oshift.labspace.studiogrizzly.com\|192.168.2.151:35551'
'\'\ \'d7f86b8193004976b330ec21728015d8\'\=\'d7f86b8193-nspace.oshift.labspace.studiogrizzly.com\|192.168.2.153:35541'
'\'--hook-nameset-proxy--cart-namehaproxy-1.4actionconnector-executecartridgeopenshift-origin-noderesult_exit_code0result_stdoutSat Jan 12 19:47:48 EET 2013: Conditionally reloading HAProxy service
result_stderr


On Fri, Jan 11, 2013 at 3:06 AM, Mike McGrath <mmcgrath redhat com> wrote:
On Thu, 10 Jan 2013, Igor Laskovy wrote:

> Hi All!
> Kindly ask clarify what logs should I look when troubleshooting scaling behavior. 
>
> I have test load by pointing two simultaneously "ab -c 11" to my test app on my local openshift lab and on haproxy-status page I can see:
>

Let us know what's working, we've got plenty of people using scaled apps
today but not a lot of feedback on it when it does/doesn't work.  Any
details about what you're doing and what you're seeing would be greatly
appreciated.

        -Mike

> stats
> Queue
> Session rate
> Sessions
> Bytes
> Denied
> Errors
> Warnings
> Server
> Cur
> Max
> Limit
> Cur
> Max
> Limit
> Cur
> Max
> Limit
> Total
> LbTot
> In
> Out
> Req
> Resp
> Req
> Conn
> Resp
> Retr
> Redis
> Status
> LastChk
> Wght
> Act
> Bck
> Chk
> Dwn
> Dwntme
> Thrtle
> Frontend
> 1
> 2
> -
> 1
> 1
> 3000
> 24
> 17766
> 263810
> 0
> 0
> 0
> OPEN
> Backend
> 0
> 0
> 0
> 0
> 0
> 0
> 3000
> 0
> 0
> 17766
> 263810
> 0
> 0
> 0
> 0
> 0
> 0
> 16m58s UP
>  
> 0
> 0
> 0
>  
> 0
>  
>
> express
> Queue
> Session rate
> Sessions
> Bytes
> Denied
> Errors
> Warnings
> Server
> Cur
> Max
> Limit
> Cur
> Max
> Limit
> Cur
> Max
> Limit
> Total
> LbTot
> In
> Out
> Req
> Resp
> Req
> Conn
> Resp
> Retr
> Redis
> Status
> LastChk
> Wght
> Act
> Bck
> Chk
> Dwn
> Dwntme
> Thrtle
> Frontend
> 0
> 8
> -
> 29
> 29
> 3000
> 57
> 45660283
> 772931274
> 0
> 0
> 0
> OPEN
> filler
> 0
> 0
> -
> 0
> 0
> 0
> 0
> -
> 0
> 0
> 0
> 0
> 0
> 0
> 0
> 0
> 0
> 1
> -
> Y
> -
> gear-d6b6cbae5e-nspace
> 0
> 0
> -
> 89
> 100
> 20
> 21
> -
> 76855
> 76832
> 24443323
> 413644074
> 0
> 0
> 0
> 0
> 0
> 16m58s UP
> L7OK/200 in 226ms
> 1
> Y
> -
> 0
> 0
> 0s
> -
> local-gear
> 0
> 0
> -
> 80
> 82
> 2
> 2
> 2
> 66722
> 66722
> 21216960
> 359287200
> 0
> 0
> 0
> 0
> 0
> 16m58s UP
> L7OK/200 in 33ms
> 1
> Y
> -
> 0
> 0
> 0s
> -
> Backend
> 0
> 0
> 169
> 180
> 22
> 23
> 3000
> 143577
> 143554
> 45660283
> 772931274
> 0
> 0
> 0
> 0
> 0
> 0
> 16m58s UP
>  
> 2
> 2
> 1
>  
> 0
> 0s
>
> --
> Igor Laskovy
> facebook.com/igor.laskovy
> Kiev, Ukraine
>
>



--
Igor Laskovy
facebook.com/igor.laskovy
Kiev, Ukraine

[Date Prev][Date Next]   [Thread Prev][Thread Next]   [Thread Index] [Date Index] [Author Index]