Setup:
Celery 4.1.0, broker=RabbitMQ 3.6.5, backend=Redis 3.2.5
Consider the following canvas:
celery worker -A worker.celeryapp:app -l info -Q default -c 2 -n defaultworker#%h -Ofair
#app.task(name='task_1',
bind=True,
base=MyConnectionHolderTask)
def task_1(self, feed_id, flow_id, **kwargs):
do_something()
task_1 = t_1.si(feed_id, flow_id)
.
.
task_13 = t_13.si(feed_id, flow_id)
(task_1 |
group((task_2 | group(task_3, task_4)),
task_5,
task_6,
task_7,
task_8) |
task_9 |
task_10 |
task_11 |
task_12 |
task_13).apply_async(link_error=unlock)
means I have chain of tasks which one of its tasks is a group of several tasks, and one of them is chain of size 2 (with latter task as group of 2).
Expected behavior
all task succeeded so expecting finish until task_13
Actual behavior
task_4 is the last to run. task_9 and the rest (10..13) doesn't run.
if i cancel the group of task_3 & task_4 it does work (till 13):
(task_1 |
group((task_2 | task_3 | task_4),
task_5,
task_6,
task_7,
task_8) |
task_9 |
task_10 |
task_11 |
task_12 |
task_13).apply_async(link_error=unlock)
Ref: Issue in github
Related
I am trying to create a "queue" system by adding an arbitrary column that creates a number based on a condition and date, to sort the importance of a row.
For example, below is the query result I pulled in Postgres:
Table: task
Result:
description | status/condition| task_created |
bla | A | 2019-12-01 07:00:00|
pikachu | A | 2019-12-01 16:32:10|
abcdef | B | 2019-12-02 18:34:22|
doremi | B | 2019-12-02 15:09:43|
lalala | A | 2019-12-03 22:10:59|
In the above, each task has a date/timestamp and status/condition applied to them. I would like to create another column that gives a number to a row where it prioritises the older tasks first, BUT if the condition is B, then we take the older task of those in B as first priority.
The expected end result (based on the example) should be:
Table1: task
description | status/condition| task_created | priority index
bla | A | 2019-12-01 07:00:00| 3
pikachu | A | 2019-12-01 16:32:10| 4
abcdef | B | 2019-12-02 18:34:22| 2
doremi | B | 2019-12-02 15:09:43| 1
lalala | A | 2019-12-03 22:10:59| 5
For priority number, 1 being most urgent to do/resolve, while 5 being the least.
How would I go about adding this additional column into the existing query? especially since there's another condition apart from just the task_created date/time.
Any help is appreciated. Many thanks!
You maybe want the Rank or Dense Rank function (depends on your needs) window functions.
If you don't need a conditional order on the status you can use this one.
SELECT *,
rank() OVER (
ORDER BY status desc, task_created
) as priority_index
FROM task
If you need a custom order based on the value of the status:
SELECT *,
rank() OVER (
ORDER BY
CASE status
WHEN 'B' THEN 1
WHEN 'A' THEN 2
WHEN 'C' THEN 3
ELSE 4
END, task_created
) as priority_index
FROM task
If you have few values this is good enough, because we can simply specify your custom order. But if you have a lot of values and the ordering information is fixed, then it should have its own table.
I'm wanting to create a graph panel in Grafana which shows the top 10 highest consumers of CPU and show their respective history over whatever time interval has been selected. I think that last part is the tricky bit.
I have this so far:
SELECT TOP("median_Percent_Processor_Time", 10) as "usage", host FROM (
SELECT median("Percent_Processor_Time") AS "median_Percent_Processor_Time" FROM "telegraf_monitoring"."autogen"."win_cpu" WHERE time > now() - 5s GROUP BY time(:interval:), "host" FILL(none)
)
This produces the following table:
time | usage | host
12/17/18 02:38:36PM | 88.4503173828125 | CNVDWSO202
12/17/18 02:38:36PM | 60.55384826660156 | CNVDSerr01
12/17/18 02:38:36PM | 46.807456970214844 | NVsABAr01
12/17/18 02:38:36PM | 27.402353286743164 | NVDARCH02
12/17/18 02:38:36PM | 21.320478439331055 | NVDABAr05
12/17/18 02:38:36PM | 5.546620845794678 | NVDALMBOE
12/17/18 02:38:36PM | 3.654918909072876 | NVDLeNCXE01
12/17/18 02:38:36PM | 47.08285903930664 | NVDOKTARAD01
The table is useful but thats just a single point in time. I need to subsequently query and pull time series data from that win_cpu measurement for those 10 hosts. The hosts values are dynamic, I have no way of predicting what will show up and because of that I cant string together OR statements and Influx doesnt support IN as far as I can see.
You can use OR regexp instead of IN. =~ /HOST1|HOST2|HOST3/ + GROUP BY host and one InfluxDB query will return all data. The tricky part is Grafana variable, which will have those top 10 hosts. When you have it, then just use advance variable formatting in the regexp query - for example =~ /${tophosts:pipe}/.
I have celery setup and working together with django. I have some periodic tasks that run. The celery log shows that the tasks are executed and that they return something.
[2017-03-26 14:34:27,039: INFO/MainProcess] Received task: my_webapp.apps.events.tasks.clean_outdated[87994396-04f7-452b-a964-f6bdd07785e0]
[2017-03-26 14:34:28,328: INFO/PoolWorker-1] Task my_webapp.apps.events.tasks.clean_outdated[87994396-04f7-452b-a964-f6bdd07785e0] succeeded in 0.05246314400005758s: 'Removed 56 event(s)
| Removed 4 SGW(s)
'
But the result is not showing up on django-celery-results admin page.
These are my settings:
CELERY_BROKER_URL = os.environ.get('BROKER_URL')
CELERY_ACCEPT_CONTENT = ['json']
CELERY_TASK_SERIALIZER = 'json'
CELERY_RESULT_SERIALIZER = 'json'
CELERY_TIMEZONE = 'Europe/Stockholm'
CELERY_RESULT_BACKEND = 'django-cache'
CELERYBEAT_SCHEDULER = 'djcelery.schedulers.DatabaseScheduler'
CELERY_RESULT_DB_SHORT_LIVED_SESSIONS = True # Fix for low traffic sites like this one
I have also tried setting CELERY_RESULT_BACKEND = 'django-db'. I know the migrations are made (when using those settings), the table exists in the database:
my_webapp=> \dt
List of relations
Schema | Name | Type | Owner
--------+--------------------------------------+-------+----------------
...
public | django_celery_beat_crontabschedule | table | my_webapp
public | django_celery_beat_intervalschedule | table | my_webapp
public | django_celery_beat_periodictask | table | my_webapp
public | django_celery_beat_periodictasks | table | my_webapp
public | django_celery_results_taskresult | table | my_webapp
...
(26 rows)
Google won't give me much help, most answers is about old libraries like djcelery. Any idea how the get the results in the table?
Here is my table in the database :
id | account_name | account_number | account_type | address | email | ifsc_code | is_default_account | phone_num | User
-----+--------------+----------------+--------------+---------+------------------------------+-----------+--------------------+-------------+----------
201 | helloi32irn | 55265766432454 | Savings | | mypal.appa99721989#gmail.com | 5545 | f | 98654567876 | abc
195 | hello | 55265766435523 | Savings | | mypal.1989#gmail.com | 5545 | t | 98654567876 | axyz
203 | what | 01010101010101 | Current | | guillaume#sample.com | 6123 | f | 09099990 | abc
On form submission in the view, which only posts a single parameter which in my case is name= "activate" which corresponds to the column "is_default_account" in the table.
I want to change the value of "is_default_account" from "t" to "f". For example here in the table, for account_name "hello" it is "t". And i want to deactivate it, i.e make it "f" and activate any of the other that has been sent trough the form
This will update your table and make account 'what' default (assuming that is_default_account is BOOLEAN field):
UPDATE table
SET is_default_account = (account_name = 'what')
You may want limit updates if table is more than just few rows you listed, like this:
UPDATE table
SET is_default_account = (account_name = 'what')
WHERE is_default_account != (account_name = 'what')
AND <limit updates by some other criteria like user name>
I think to accomplish what you want to do you should send at least two values from the form. One for the id of the account you want to update and the other for the action (activate here). You can also just send the id and have it toggle. There are many ways to do this but I can't figure out exactly what you are trying to do and whether you want SQL or Playframework code. Without limiting your update in somewhere (like id) you can't precisely control what specific rows get updated. Please clarify your question and add some more code if you want help on the playframework side, which I would think you do.
Where can I find the documentation for MicroStrategy Command Manager? I've look through various docs that I have but not able to find any comprehensive list of commands. Particularly, I need to know a list of commands to create attributes and metrics.
Thanks
The best way to find the list of available commands is to use the Outlines option, available inside Command Manager.
To create an attribute use the following syntax:
CREATE ATTRIBUTE "<attribute_name>" [DESCRIPTION "<description>"] [LONGDESCRIPTION "<long_description>"] IN [FOLDER] "<location_path>" [HIDDEN (TRUE | FALSE)] ATTRIBUTEFORM "<form_name>" [FORMCATEGORY "<category_name>"] [FORMDESC "<form_description>"] [FORMTYPE (NUMBER | TEXT | DATETIME | DATE | TIME | URL | EMAIL | HTML | PICTURE | BIGDECIMAL | PHONENUMBER)] [REPORTSORT (NONE | ASC | DESC)] [BROWSESORT (NONE | ASC | DESC)] EXPRESSION "<form_expression>" [MAPPINGMODE (AUTOMATIC | MANUAL)] [EXPSOURCETABLES "<sourcetable1>" [, "<sourcetable2>" ...]] LOOKUPTABLE "<lookup_table>" FOR PROJECT "<project_name>";
and for metrics creation:
CREATE METRIC "<metric_name>" IN [FOLDER] "<location_path>" EXPRESSION "<expression>" [DESCRIPTION "<description>"] [LONGDESCRIPTION "<long_description>"] [HIDDEN (TRUE | FALSE)] [ALLOWSMARTMETRIC (TRUE | FALSE)] [REMOVEREPORTFILTERELEMENTS (TRUE | FALSE)] [TOTALSUBTOTALFUNCTION (AVERAGE | COUNT | DEFAULT| GEOMETRICMEAN | MAXIMUM | MEDIAN | MINIMUM | MODE | NONE | PRODUCT | STANDARDDEVIATION | SUM | VARIANCE)] [DYNAMICAGGREGATIONFUNCTION (AVERAGE | COUNT | DEFAULT| GEOMETRICMEAN | MAXIMUM | MEDIAN | MINIMUM | MODE | NONE | PRODUCT | STANDARDDEVIATION | SUM | VARIANCE)] [COLUMNALIAS "<columnalias>"] FOR PROJECT "<project_name>";
Some simple examples provided bellow (again from the outlines available inside command manager):
CREATE ATTRIBUTE "Day" DESCRIPTION "Duplicate of Day Attribute from folder \Time" IN FOLDER "\Schema Objects\Attributes" ATTRIBUTEFORM "ID" FORMCATEGORY "ID" FORMDESC "Basic ID form" FORMTYPE TEXT REPORTSORT ASC EXPRESSION "[DAY_DATE]" LOOKUPTABLE "LU_DAY" FOR PROJECT "MicroStrategy Tutorial";
CREATE METRIC "New Metric" IN FOLDER "\Public Objects\Metrics\Count Metrics" EXPRESSION "Count(Customer) {Country} <[Western United States Customers]>" FOR PROJECT "MicroStrategy Tutorial";
As Bruno said, the Outlines option gives you all the templates that there are in Command Manager, which you can build together and define to do what you want. If in doubt, check out the "MicroStrategy System Administration Guide" product manual, which covers all you need to know on command manager.
Here is one useful site with Command Manager documentation:
https://metacpan.org/pod/Business::Intelligence::MicroStrategy::CommandManager