Not able to create chain tasks in parallel tasks

[baloda]

I am trying to create a below-defined workflow and it is giving error

p.TEST:
  tasks:
    - TEST_1
    - TEST_2
    - WORKERS:
        type: group
        tasks:
          - TEST_3
          - TEST3_WORKERS:
            type: chain
            tasks:
              - TEST_6
              - TEST_7
          - TEST_4
          - TEST_5
    - TEST_8

Error:

signature = cel.tasks.get(task_name).subtask(
TypeError: unhashable type: dict

[ncrocfer]

Director does not yet support nested structure because at the origin we wanted to keep our workflows as simple as possible.
But looking at your use-case is a proof that it can be useful sometimes :)

We'll work to implement that in a next release.

[henri42]

Hi, thanks for this nice project !

I might be interested by this feature.
Is it still planned to be developed ?

[ncrocfer]

Hello,

No we don't plan to add it right now because we still want to keep the workflows as simple as possible. Can you explain us what's your use-case please?

[henri42]

Thanks for your answer.

My use case would be the following:

• detecting media files
• analyze these files
• processing files differently depending on the result of the analysis (this would typically be a chain of tasks)

A related question : In order to build more complex process, how do you suggest to chain the simple worfklows ? Maybe an external application using the rest API ?

[kevinnguyen9185]

Hi, thks for great project!

I'm wondering if we have put it into some milesonte?

[kevinnguyen9185]

Hi @ncrocfer I've made some changes from my local branch, which now makes celery-director fully support nested workflow. What do you think?

[ncrocfer]

Hi @cin-kevin,

Indeed this seems interesting 😊

The main goal of Celery Director is to keep the definitions as simple as possible. If your implementation of this feature doesn't complexify the YAML definition or the codebase you can make a pull-request we would be happy to review it 👍

[kevinnguyen9185]

Thks @ncrocfer I'll submit a pull-request !