[bounty] build one end to end simple workflow that works with 90% success rate

[louis030195]

example:

• go to x.com and scroll my feed and follow everyone you see

using the execute_sequence tool

so you should share a .json file that we can input into the tool and would do the automation

rule of thumb for your PR quality:

• if I can reproduce your work in less than 3 prompts you shouldn't send a PR

add video in the PR showing it works

/bounty 100

[algora-pbc]

💎 $100 bounty • screenpi.pe

Steps to solve:

1. Start working: Comment /attempt #206 with your implementation plan
2. Submit work: Create a pull request including /claim #206 in the PR body to claim the bounty
3. Receive payment: 100% of the bounty is received 2-5 days post-reward. Make sure you are eligible for payouts

❗ Important guidelines:

• To claim a bounty, you need to provide a short demo video of your changes in your pull request
• If anything is unclear, ask for clarification before starting as this will help avoid potential rework
• Low quality AI PRs will not receive review and will be closed
• Do not ask to be assigned unless you've contributed before

Thank you for contributing to mediar-ai/terminator!

Attempt	Started (UTC)	Solution	Actions
🟢 @tribhuwan-kumar	Aug 07, 2025, 05:16:55 AM	#242	Reward
🟢 @feliciien	Jul 15, 2025, 06:50:01 PM	#207	Reward
🟢 @MAVRICK-1	Jul 16, 2025, 01:42:44 AM	WIP
🟢 @dfordp	Jul 16, 2025, 05:58:41 AM	#213	Reward
🟢 @iamsyg	Jul 27, 2025, 08:34:35 PM	#236	Reward

[louis030195]

feel free to ask questions if need more context (instead of building the wrong thing)

[louis030195]

examples

{
  "tool_name": "execute_sequence",
  "arguments": {
    "variables": {
      "url":               { "type": "string", "label": "URL",               "default": "https://example.demoapp.com/" },
      "products_to_enable":{ "type": "array",  "label": "Products to Enable","default": ["Product Alpha","Product Beta"] },
      "products_parser":   {
        "type": "object",
        "label": "Products State Parser",
        "default": {
          "itemContainerDefinition": { "nodeConditions":[ { "property":"role","op":"equals","value":"CheckBox" } ] },
          "fieldsToExtract": {
            "productName": { "extractProperty": "name" },
            "isEnabled":   { "extractProperty": "is_selected" }
          }
        }
      }
    },

    "inputs": {
      "url": "https://example.demoapp.com/",
      "products_to_enable": ["Product Alpha","Product Beta"]
    },

    "selectors": {
      "app_window":                  "role:Window|name:Demo App",
      "settings_button":             "role:Text|name:Settings",
      "set_available_products_btn":  "role:Text|name:Set Available Products",
      "auto_saved_text":             "role:Text|name:Changes are auto saved"
    },

    "steps": [
      /* 1️⃣ Launch */
      { "tool_name": "navigate_browser", "arguments": { "url": "{{url}}", "include_tree": false } },

      /* 2️⃣ Open settings */
      {
        "group_name": "Open Product Settings",
        "steps": [
          { "tool_name":"click_element", "arguments":{ "selector":"{{selectors.settings_button}}","timeout_ms":1500,"include_tree":false } },
          { "tool_name":"click_element", "arguments":{ "selector":"{{selectors.set_available_products_btn}}","timeout_ms":1500,"include_tree":false } },
          { "tool_name":"wait_for_element","arguments":{ "selector":"{{selectors.auto_saved_text}}","condition":"visible","timeout_ms":5000,"include_tree":false } }
        ]
      },

      /* 3️⃣ Toggle sample products */
      { "tool_name":"set_toggled","arguments":{ "selector":"role:CheckBox|name: Product Alpha","state":"{{contains(products_to_enable,'Product Alpha')}}","include_tree":false } },
      { "tool_name":"set_toggled","arguments":{ "selector":"role:CheckBox|name: Product Beta","state":"{{contains(products_to_enable,'Product Beta')}}","include_tree":false } },

      /* 4️⃣ Capture result with parser */
      {
        "tool_name":"wait_for_output_parser",
        "arguments":{
          "output_parser":"{{products_parser}}",
          "timeout_ms":5000,
          "poll_interval_ms":1000,
          "success_criteria":{ "min_items":1,"required_fields":["productName","isEnabled"] },
          "include_tree":true
        }
      },

      /* 5️⃣ Close app (always runs) */
      {
        "group_name":"Cleanup",
        "if":"always()",
        "steps":[
          { "tool_name":"close_element","arguments":{ "selector":"{{selectors.app_window}}","include_tree":false } }
        ]
      }
    ],

    "output_parser": "{{products_parser}}"
  }
}

[feliciien]

/attempt #206

🛠️ Implementation Plan

I will build an end-to-end workflow using the execute_sequence tool that:

1. Navigates to https://x.com
2. Waits for the feed to load
3. Scrolls the feed multiple times
4. Clicks the "Follow" button on all visible user cards (up to 5)
5. Ends the sequence

🎯 Goal: Automate a basic user interaction that can be used as a reusable demo

🔧 Output: A single .json file that can be loaded directly into the tool

🎥 A short video demo will be included in the PR to showcase the execution.

[feliciien]

@louis030195 Quick question before I proceed —

I'm working on macOS, and I noticed from the documentation that UI automation (especially execute_sequence and tools like click_element, set_toggled, etc.) are only fully supported on Windows.

❓ Is it possible to fully test and submit a workflow (like in #206) on macOS, or is Windows mandatory to execute and record it properly?

I want to make sure I don’t waste time building a flow that can’t be executed on my OS. Thanks in advance!

[feliciien]

👋 I want to implement the following workflow:

• Open https://x.com in the browser
• Scroll down the feed
• Automatically click all visible "Follow" buttons
• Repeat scroll + follow to reach more users

This will be done using the execute_sequence tool with selectors like role:Button|name:Follow.

Note: I'm currently on macOS, so I cannot run the workflow locally to fully test it (UI automation is limited). But I will submit a valid .json workflow with proper structure and selectors. I'd appreciate if someone on Windows can confirm the behavior.

Will follow up with a PR shortly.

[MAVRICK-1]

/attempt #206

[dfordp]

/attempt #206

[louis030195]

some recs to build this kind of thing:

• install cursor
• install terminator mcp in cursor
• ask to do an automation (eg go to whatsapp scroll all message, and add to apple note ) idk
• then ask to export as a yaml sequence with this example:

tool_name: execute_sequence
arguments:
  variables:
    url:
      type: string
      label: URL
      default: "https://web.chatterbox.io"

    notifications_to_enable:
      type: array
      label: Notifications to Enable
      default:
        - "Message Previews"
        - "Typing Indicators"

    notifications_parser:
      type: object
      label: Notifications Parser
      default:
        itemContainerDefinition:
          nodeConditions:
            - property: role
              op: equals
              value: CheckBox
        fieldsToExtract:
          label:
            extractProperty: name
          isEnabled:
            extractProperty: is_selected

  inputs:
    url: "https://web.chatterbox.io"
    notifications_to_enable:
      - "Message Previews"
      - "Typing Indicators"

  selectors:
    app_window: "role:Window|name:ChatterBox"
    settings_button: "role:Text|name:Settings"
    notifications_tab: "role:Text|name:Notifications"
    saved_banner: "role:Text|name:Settings saved successfully"

  steps:
    - tool_name: navigate_browser
      arguments:
        url: "${{url}}"
        include_tree: false

    - group_name: "Navigate to Notification Settings"
      steps:
        - tool_name: click_element
          arguments:
            selector: "${{selectors.settings_button}}"
            timeout_ms: 1500
            include_tree: false
        - tool_name: click_element
          arguments:
            selector: "${{selectors.notifications_tab}}"
            timeout_ms: 1500
            include_tree: false
        - tool_name: wait_for_element
          arguments:
            selector: "${{selectors.saved_banner}}"
            condition: "visible"
            timeout_ms: 3000
            include_tree: false

    - group_name: "Set Notification Toggles"
      steps:
        - tool_name: set_toggled
          arguments:
            selector: "role:CheckBox|name:Message Previews"
            state: "${{contains(notifications_to_enable,'Message Previews')}}"
            include_tree: false

        - tool_name: set_toggled
          arguments:
            selector: "role:CheckBox|name:Typing Indicators"
            state: "${{contains(notifications_to_enable,'Typing Indicators')}}"
            include_tree: false

    - group_name: "Capture Current Notification Settings"
      steps:
        - tool_name: wait_for_output_parser
          arguments:
            output_parser: "${{notifications_parser}}"
            timeout_ms: 8000
            poll_interval_ms: 1000
            success_criteria:
              min_items: 1
              required_fields: ["label", "isEnabled"]
            include_tree: true

    - group_name: "Close ChatterBox App"
      if: "always()"
      steps:
        - tool_name: close_element
          arguments:
            selector: "${{selectors.app_window}}"
            include_tree: false

  output_parser: "${{notifications_parser}}"

then you can execute the yaml using

cargo build --release
./target/release/gist-executor chatterbox-notification-setup.yaml

if it works, good, otherwise keep iterating

you can also ask AI in cursor to execute the sequence (using execute_sequence tool)