Discussion:
[pmacct-discussion] losing environment variables when setting sql_startup_delay
Johannes Maybaum
2018-03-08 02:04:24 UTC
Permalink
Hi,

I am having a problem with the environment variables that is run after
the cache is pruged(sql_trigger_exec). As soon as I set
sql_startup_delay, some environment variables are not set anymore.

sql_startup_delay not set:

set | grep SQL ->
EFFECTIVE_SQL_TABLE=acct_20180308_02
SQL_ACTIVE_WRITERS=0
SQL_DB=pmacct
SQL_HISTORY_BASETIME=1520472300
SQL_HISTORY_TIMESLOT=300
SQL_MAX_WRITERS=10
SQL_REFRESH_TIME=300
SQL_TABLE=acct_%Y%m%d_%H
SQL_USER=pmacct


sql_startup_delay set:

set | grep SQL ->
SQL_DB=pmacct
SQL_MAX_WRITERS=10
SQL_REFRESH_TIME=300
SQL_TABLE=acct_%Y%m%d_%H
SQL_USER=pmacct


Has anybody else seen this problem?


thanks,

Johannes
--
Johannes Maybaum
2018-03-08 11:05:48 UTC
Permalink
Forgot to mention I am using pmacctd to read accounting data from
several interfaces to be stored in a mysql db for further processing.
I see this effect in pmacct v1.6.1 as well as in v1.7.0.
Obviously the script started using sql_trigger_exec is running into some
problems... ;-)
Post by Johannes Maybaum
Hi,
I am having a problem with the environment variables that is run after
the cache is pruged(sql_trigger_exec). As soon as I set
sql_startup_delay, some environment variables are not set anymore.
set | grep SQL ->
EFFECTIVE_SQL_TABLE=acct_20180308_02
SQL_ACTIVE_WRITERS=0
SQL_DB=pmacct
SQL_HISTORY_BASETIME=1520472300
SQL_HISTORY_TIMESLOT=300
SQL_MAX_WRITERS=10
SQL_REFRESH_TIME=300
SQL_TABLE=acct_%Y%m%d_%H
SQL_USER=pmacct
set | grep SQL ->
SQL_DB=pmacct
SQL_MAX_WRITERS=10
SQL_REFRESH_TIME=300
SQL_TABLE=acct_%Y%m%d_%H
SQL_USER=pmacct
Has anybody else seen this problem?
thanks,
Johannes
_______________________________________________
pmacct-discussion mailing list
http://www.pmacct.net/#mailinglists
Paolo Lucente
2018-03-09 14:54:33 UTC
Permalink
Hi Johannes,

Do you see that behaviour - the reduced amount of environment variables
being set - in coincidence with empty purge events? That is zero entries
pushed to the database? If yes: that actually was intended behaviour and
it still kind of makes sense to me; maybe i would refine it by still
setting the SQL_ACTIVE_WRITERS variable. Would you see things in a
different way? If no: then it's a bug and we can follow-up by unicast
email since i was unable to reproduce it.

Paolo
Post by Johannes Maybaum
Forgot to mention I am using pmacctd to read accounting data from
several interfaces to be stored in a mysql db for further processing.
I see this effect in pmacct v1.6.1 as well as in v1.7.0.
Obviously the script started using sql_trigger_exec is running into some
problems... ;-)
Post by Johannes Maybaum
Hi,
I am having a problem with the environment variables that is run after
the cache is pruged(sql_trigger_exec). As soon as I set
sql_startup_delay, some environment variables are not set anymore.
set | grep SQL ->
EFFECTIVE_SQL_TABLE=acct_20180308_02
SQL_ACTIVE_WRITERS=0
SQL_DB=pmacct
SQL_HISTORY_BASETIME=1520472300
SQL_HISTORY_TIMESLOT=300
SQL_MAX_WRITERS=10
SQL_REFRESH_TIME=300
SQL_TABLE=acct_%Y%m%d_%H
SQL_USER=pmacct
set | grep SQL ->
SQL_DB=pmacct
SQL_MAX_WRITERS=10
SQL_REFRESH_TIME=300
SQL_TABLE=acct_%Y%m%d_%H
SQL_USER=pmacct
Has anybody else seen this problem?
thanks,
Johannes
_______________________________________________
pmacct-discussion mailing list
http://www.pmacct.net/#mailinglists
_______________________________________________
pmacct-discussion mailing list
http://www.pmacct.net/#mailinglists
Johannes Maybaum
2018-03-09 21:52:12 UTC
Permalink
Hi Paolo,


I see this behaviour with non empty purge events.

Here is a print out of all variables listet in the TRIGGER_VARS file as
seen by the script:

sql_startup_delay == 0

SQL_DB = pmacct
SQL_TABLE = acct_%Y%m%d
EFFECTIVE_SQL_TABLE = acct_20180309
SQL_HOST =
SQL_USER = pmacct
SQL_REFRESH_TIME = 300
SAMPLING_RATE =
SQL_RECOVERY_BACKUP_HOST =
TOTAL_ELEM_NUMBER = 4649
EFFECTIVE_ELEM_NUMBER = 4622
INSERT_QUERIES_NUMBER = 1
UPDATE_QUERIES_NUMBER = 0
ELAPSED_TIME = 1
SQL_HISTORY_BASETIME = 1520631900
SQL_HISTORY_TIMESLOT = 300
SQL_MAX_WRITERS = 10
SQL_ACTIVE_WRITERS = 0


sql_startup_delay != 0

SQL_DB = pmacct
SQL_TABLE = acct_%Y%m%d
EFFECTIVE_SQL_TABLE =
SQL_HOST =
SQL_USER = pmacct
SQL_REFRESH_TIME = 300
SAMPLING_RATE =
SQL_RECOVERY_BACKUP_HOST =
TOTAL_ELEM_NUMBER =
EFFECTIVE_ELEM_NUMBER =
INSERT_QUERIES_NUMBER =
UPDATE_QUERIES_NUMBER =
ELAPSED_TIME =
SQL_HISTORY_BASETIME =
SQL_HISTORY_TIMESLOT =
SQL_MAX_WRITERS = 10
SQL_ACTIVE_WRITERS =



What would be the recommended way for the script to check if the purge
event was empty? My script would have probably failed on an empty purge
event.

What information do you need from me to be able to reproduce the problem?


Johannes
--
Post by Paolo Lucente
Hi Johannes,
Do you see that behaviour - the reduced amount of environment variables
being set - in coincidence with empty purge events? That is zero entries
pushed to the database? If yes: that actually was intended behaviour and
it still kind of makes sense to me; maybe i would refine it by still
setting the SQL_ACTIVE_WRITERS variable. Would you see things in a
different way? If no: then it's a bug and we can follow-up by unicast
email since i was unable to reproduce it.
Paolo
Post by Johannes Maybaum
Forgot to mention I am using pmacctd to read accounting data from
several interfaces to be stored in a mysql db for further processing.
I see this effect in pmacct v1.6.1 as well as in v1.7.0.
Obviously the script started using sql_trigger_exec is running into some
problems... ;-)
Post by Johannes Maybaum
Hi,
I am having a problem with the environment variables that is run after
the cache is pruged(sql_trigger_exec). As soon as I set
sql_startup_delay, some environment variables are not set anymore.
set | grep SQL ->
EFFECTIVE_SQL_TABLE=acct_20180308_02
SQL_ACTIVE_WRITERS=0
SQL_DB=pmacct
SQL_HISTORY_BASETIME=1520472300
SQL_HISTORY_TIMESLOT=300
SQL_MAX_WRITERS=10
SQL_REFRESH_TIME=300
SQL_TABLE=acct_%Y%m%d_%H
SQL_USER=pmacct
set | grep SQL ->
SQL_DB=pmacct
SQL_MAX_WRITERS=10
SQL_REFRESH_TIME=300
SQL_TABLE=acct_%Y%m%d_%H
SQL_USER=pmacct
Has anybody else seen this problem?
thanks,
Johannes
_______________________________________________
pmacct-discussion mailing list
http://www.pmacct.net/#mailinglists
_______________________________________________
pmacct-discussion mailing list
http://www.pmacct.net/#mailinglists
_______________________________________________
pmacct-discussion mailing list
http://www.pmacct.net/#mailinglists
Paolo Lucente
2018-03-12 13:08:19 UTC
Permalink
Hi Johannes,

Actually in this specific case the best option would be access to your
system, if it's non production. I'm puzzled because some varbiables, ie.
SQL_HISTORY_BASETIME, are set only if there is a valid value for it so,
according to the code, it's not an option to have it declared but empty.

Did you take the whole list from TRIGGER_VARS and fill with empties the
ones you do not see populated for the purpose of the email? That would
make sense but then we return to the theory of the empty purge event;
you can manually check if that was the case by looking in the logs for
the following strings:

*** Purging cache - START (PID: %u) ***
*** Purging cache - END (PID: %u, QN: 0/0, ET: 0) ***

If the theory is confirmed i agree that the setting of env variables for
such case can be improved, ie. set EFFECTIVE_ELEM_NUMBER to zero
(instead of not setting it at all).

Paolo
Post by Johannes Maybaum
Hi Paolo,
I see this behaviour with non empty purge events.
Here is a print out of all variables listet in the TRIGGER_VARS file as
sql_startup_delay == 0
SQL_DB = pmacct
SQL_TABLE = acct_%Y%m%d
EFFECTIVE_SQL_TABLE = acct_20180309
SQL_HOST =
SQL_USER = pmacct
SQL_REFRESH_TIME = 300
SAMPLING_RATE =
SQL_RECOVERY_BACKUP_HOST =
TOTAL_ELEM_NUMBER = 4649
EFFECTIVE_ELEM_NUMBER = 4622
INSERT_QUERIES_NUMBER = 1
UPDATE_QUERIES_NUMBER = 0
ELAPSED_TIME = 1
SQL_HISTORY_BASETIME = 1520631900
SQL_HISTORY_TIMESLOT = 300
SQL_MAX_WRITERS = 10
SQL_ACTIVE_WRITERS = 0
sql_startup_delay != 0
SQL_DB = pmacct
SQL_TABLE = acct_%Y%m%d
EFFECTIVE_SQL_TABLE =
SQL_HOST =
SQL_USER = pmacct
SQL_REFRESH_TIME = 300
SAMPLING_RATE =
SQL_RECOVERY_BACKUP_HOST =
TOTAL_ELEM_NUMBER =
EFFECTIVE_ELEM_NUMBER =
INSERT_QUERIES_NUMBER =
UPDATE_QUERIES_NUMBER =
ELAPSED_TIME =
SQL_HISTORY_BASETIME =
SQL_HISTORY_TIMESLOT =
SQL_MAX_WRITERS = 10
SQL_ACTIVE_WRITERS =
What would be the recommended way for the script to check if the purge
event was empty? My script would have probably failed on an empty purge
event.
What information do you need from me to be able to reproduce the problem?
Johannes
--
Post by Paolo Lucente
Hi Johannes,
Do you see that behaviour - the reduced amount of environment variables
being set - in coincidence with empty purge events? That is zero entries
pushed to the database? If yes: that actually was intended behaviour and
it still kind of makes sense to me; maybe i would refine it by still
setting the SQL_ACTIVE_WRITERS variable. Would you see things in a
different way? If no: then it's a bug and we can follow-up by unicast
email since i was unable to reproduce it.
Paolo
Post by Johannes Maybaum
Forgot to mention I am using pmacctd to read accounting data from
several interfaces to be stored in a mysql db for further processing.
I see this effect in pmacct v1.6.1 as well as in v1.7.0.
Obviously the script started using sql_trigger_exec is running into some
problems... ;-)
Post by Johannes Maybaum
Hi,
I am having a problem with the environment variables that is run after
the cache is pruged(sql_trigger_exec). As soon as I set
sql_startup_delay, some environment variables are not set anymore.
set | grep SQL ->
EFFECTIVE_SQL_TABLE=acct_20180308_02
SQL_ACTIVE_WRITERS=0
SQL_DB=pmacct
SQL_HISTORY_BASETIME=1520472300
SQL_HISTORY_TIMESLOT=300
SQL_MAX_WRITERS=10
SQL_REFRESH_TIME=300
SQL_TABLE=acct_%Y%m%d_%H
SQL_USER=pmacct
set | grep SQL ->
SQL_DB=pmacct
SQL_MAX_WRITERS=10
SQL_REFRESH_TIME=300
SQL_TABLE=acct_%Y%m%d_%H
SQL_USER=pmacct
Has anybody else seen this problem?
thanks,
Johannes
_______________________________________________
pmacct-discussion mailing list
http://www.pmacct.net/#mailinglists
_______________________________________________
pmacct-discussion mailing list
http://www.pmacct.net/#mailinglists
_______________________________________________
pmacct-discussion mailing list
http://www.pmacct.net/#mailinglists
Loading...