Custom Functions & Tools

Custom Functions

Browserable provides a flexible way to extend its functionality through custom tools and functions. This allows you to add specialized capabilities to your browser automation tasks.

Using Tool Calls with REST API

Here’s a step-by-step guide on how to use tool calls in your automation tasks:

1. Create a Task with Tool Call Support

Example: Let’s create a task that requires user inputFirst:

curl -X POST https://api.browserable.ai/api/v1/task/create \
  -H "x-api-key: YOUR_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "task": "Log into my account",
    "agent": "BROWSER_AGENT",
    "tools": [
      {
        "type": "function",
        "function": {
          "name": "form_input",
          "description": "Get user input for login credentials",
          "parameters": {
            "type": "object",
            "properties": {
              "heading": {
                "type": "string",
                "description": "Form heading"
              },
              "description": {
                "type": "string",
                "description": "Form description"
              },
              "fields": {
                "type": "array",
                "items": {
                  "type": "object",
                  "properties": {
                    "type": { "type": "string" },
                    "label": { "type": "string" },
                    "placeholder": { "type": "string" }
                  }
                }
              }
            }
          }
        }
      }
    ]
  }'

Note that your tools can look widely different. As long as they are of type ‘function’, you can change the properties (to any json schema), name and decription of the tools according to your usecase.

2. Check Task Status

Monitor the task status to detect when tool is called:

curl -X GET https://api.browserable.ai/api/v1/task/TASK_ID/run/status \
  -H "x-api-key: YOUR_API_KEY"

Response when waiting for tool input:

{
  "success": true,
  "data": {
    "status": "running",
    "detailedStatus": "tool_call",
    "toolCall": {
      "toolCallId": "unique string to identify this call",
      "tool": {
        "name": "form_input", // this will be the tool function name you have provided
        "parameters": {
          // Tool call parameters in line with your definition
        }
      }
    }
  }
}

3. Submit Tool Input

When a tool call is detected, submit the user input (only string inputs are allowed):

curl -X POST https://api.browserable.ai/api/v1/task/TASK_ID/run/tool-input \
  -H "x-api-key: YOUR_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "input": "[{\"label\":\"Username\",\"value\":\"user@example.com\"},{\"label\":\"Password\",\"value\":\"password123\"}]",
    "toolCallId": "TOOL_CALL_ID"
  }'

Form Tool Call Example

The form tool call is a powerful feature that allows you to collect structured input from users. Here’s a complete example:

1. Task Creation with Form Tool

const createTaskWithForm = async () => {
  const response = await fetch(
    "https://api.browserable.ai/api/v1/task/create",
    {
      method: "POST",
      headers: {
        "x-api-key": "YOUR_API_KEY",
        "Content-Type": "application/json",
      },
      body: JSON.stringify({
        task: "Login to website",
        agent: "BROWSER_AGENT",
        tools: [
          {
            type: "function",
            function: {
              name: "form_input",
              description: `Use this tool to ask information from the user. This is the only way to get information from the user. And assume that user might not have any other immediate context. So any relevant/ related information user needs can be provided to the user using this.
When to use this:
- If you are stuck and you need help from user
- Captchas that you are not able to solve yourself
- Logins, Signups etc.`,
              parameters: {
                type: "object",
                properties: {
                  heading: {
                    type: "string",
                    description: "The heading of the form input",
                  },
                  description: {
                    type: "string",
                    description: "The description of the form input",
                  },
                  fields: {
                    type: "array",
                    items: {
                      type: "object",
                      properties: {
                        type: {
                          type: "string",
                          description:
                            "What type of input to show to the user",
                          enum: [
                            "text",
                            "number",
                            "email",
                            "tel",
                            "url",
                            "password",
                            "date",
                            "time",
                            "datetime-local",
                            "month",
                            "week",
                            "textarea",
                            "select",
                            "checkbox",
                            "radio",
                            "switch",
                            "range",
                          ],
                        },
                        label: {
                          type: "string",
                          description: "Label text for the input field",
                        },
                        placeholder: {
                          type: "string",
                          description:
                            "Placeholder text shown when field is empty",
                        },
                        defaultValue: {
                          type: ["string", "number", "boolean"],
                          description: "Default value for the field",
                        },
                        readonly: {
                          type: "boolean",
                          description: "Whether the field is read-only",
                          default: false,
                        },
                        required: {
                          type: "boolean",
                          description: "Whether the field is required",
                          default: false,
                        },
                        disabled: {
                          type: "boolean",
                          description: "Whether the field is disabled",
                          default: false,
                        },
                        min: {
                          type: "number",
                          description:
                            "Minimum value for number/range/date inputs",
                        },
                        max: {
                          type: "number",
                          description:
                            "Maximum value for number/range/date inputs",
                        },
                        step: {
                          type: "number",
                          description: "Step value for number/range inputs",
                        },
                        minLength: {
                          type: "number",
                          description: "Minimum length for text inputs",
                        },
                        maxLength: {
                          type: "number",
                          description: "Maximum length for text inputs",
                        },
                        pattern: {
                          type: "string",
                          description:
                            "Regular expression pattern for validation",
                        },
                        options: {
                          type: "array",
                          description:
                            "Options for select/radio/checkbox inputs",
                          items: {
                            type: "object",
                            properties: {
                              label: {
                                type: "string",
                                description: "Display label for the option",
                              },
                              value: {
                                type: ["string", "number"],
                                description: "Value for the option",
                              },
                            },
                          },
                        },
                        multiple: {
                          type: "boolean",
                          description:
                            "Allow multiple selections for select input",
                          default: false,
                        },
                        rows: {
                          type: "number",
                          description: "Number of rows for textarea",
                        },
                        cols: {
                          type: "number",
                          description: "Number of columns for textarea",
                        },
                        autocomplete: {
                          type: "string",
                          description: "Autocomplete attribute value",
                        },
                        className: {
                          type: "string",
                          description:
                            "CSS class names to apply to the input",
                        },
                      },
                      required: ["type"],
                    },
                  },
                  submitButton: {
                    type: "object",
                    properties: {
                      label: {
                        type: "string",
                        description: "Label text for the submit button",
                      },
                    },
                  },
                },
              },
            },
          },
        ],
      }),
    }
  );
  return await response.json();
};

2. Example Form Parameters

When the agent needs user input, it will call the form tool with parameters like:

{
  "heading": "Login Credentials",
  "description": "Please enter your login credentials",
  "fields": [
    {
      "type": "text",
      "label": "Username",
      "placeholder": "Enter your username",
      "required": true
    },
    {
      "type": "password",
      "label": "Password",
      "placeholder": "Enter your password",
      "required": true
    },
    {
      "type": "checkbox",
      "label": "Remember me",
      "placeholder": "Keep me logged in",
      "required": false
    }
  ],
  "submitButton": {
    "label": "Login"
  }
}

3. Rendering the Form

You can render the form according to your UI preferences. Here’s one simple implementation:

import React, { useState } from 'react';
import TextareaAutosize from 'react-textarea-autosize';

const FormInput = ({ formData, onSubmit }) => {
  const [formState, setFormState] = useState(
    formData.fields.reduce((acc, field) => {
      acc[field.label] = field.defaultValue || '';
      return acc;
    }, {})
  );

  const handleInputChange = (label, value) => {
    setFormState(prev => ({
      ...prev,
      [label]: value
    }));
  };

  const handleSubmit = (e) => {
    e.preventDefault();
    const formattedData = formData.fields.map(field => ({
      ...field,
      value: formState[field.label]
    }));
    onSubmit(formattedData);
  };

  const renderField = (field) => {
    const commonClasses = "w-full px-3 py-1 text-sm border border-gray-300 rounded-md focus:outline-none focus:ring-2 focus:ring-blue-500 focus:border-transparent";

    switch (field.type) {
      case 'textarea':
        return (
          <TextareaAutosize
            minRows={field.rows || 3}
            maxRows={field.maxRows || 10}
            className={commonClasses}
            placeholder={field.placeholder}
            value={formState[field.label] || ''}
            onChange={(e) => handleInputChange(field.label, e.target.value)}
            disabled={field.disabled}
            readOnly={field.readonly}
            required={field.required}
          />
        );

      case 'select':
        return (
          <select
            className={commonClasses}
            value={formState[field.label] || ''}
            onChange={(e) => handleInputChange(field.label, e.target.value)}
            disabled={field.disabled}
            required={field.required}
            multiple={field.multiple}
          >
            <option value="">Select an option</option>
            {field.options?.map((option, idx) => (
              <option key={idx} value={option.value}>
                {option.label}
              </option>
            ))}
          </select>
        );

      case 'checkbox':
        return (
          <div className="flex items-center">
            <input
              type="checkbox"
              className="h-4 w-4 text-sm text-blue-600 focus:ring-blue-500 border-gray-300 rounded"
              checked={formState[field.label] || false}
              onChange={(e) => handleInputChange(field.label, e.target.checked)}
              disabled={field.disabled}
              required={field.required}
            />
            <span className="ml-2 text-sm text-gray-600">{field.placeholder}</span>
          </div>
        );

      case 'radio':
        return (
          <div className="space-y-2">
            {field.options?.map((option, idx) => (
              <div key={idx} className="flex items-center">
                <input
                  type="radio"
                  className="h-4 w-4 text-blue-600 focus:ring-blue-500 border-gray-300"
                  name={field.label}
                  value={option.value}
                  checked={formState[field.label] === option.value}
                  onChange={(e) => handleInputChange(field.label, e.target.value)}
                  disabled={field.disabled}
                  required={field.required}
                />
                <span className="ml-2 text-sm text-gray-600">{option.label}</span>
              </div>
            ))}
          </div>
        );

      case 'switch':
        return (
          <label className="inline-flex items-center cursor-pointer">
            <input
              type="checkbox"
              className="sr-only peer"
              checked={formState[field.label] || false}
              onChange={(e) => handleInputChange(field.label, e.target.checked)}
              disabled={field.disabled}
            />
            <div className="relative w-11 h-6 bg-gray-200 peer-focus:outline-none peer-focus:ring-4 peer-focus:ring-blue-300 rounded-full peer peer-checked:after:translate-x-full peer-checked:after:border-white after:content-[''] after:absolute after:top-[2px] after:left-[2px] after:bg-white after:border-gray-300 after:border after:rounded-full after:h-5 after:w-5 after:transition-all peer-checked:bg-blue-600"></div>
            <span className="ml-3 text-sm font-medium text-gray-700">{field.placeholder}</span>
          </label>
        );

      case 'range':
        return (
          <div className="flex flex-col">
            <input
              type="range"
              className="w-full h-2 bg-gray-200 rounded-lg appearance-none cursor-pointer"
              min={field.min}
              max={field.max}
              step={field.step}
              value={formState[field.label] || field.min || 0}
              onChange={(e) => handleInputChange(field.label, e.target.value)}
              disabled={field.disabled}
            />
            <span className="text-sm text-gray-500 mt-1">
              Value: {formState[field.label] || field.min || 0}
            </span>
          </div>
        );

      default:
        return (
          <input
            type={field.type}
            className={commonClasses}
            placeholder={field.placeholder}
            value={formState[field.label] || ''}
            onChange={(e) => handleInputChange(field.label, e.target.value)}
            min={field.min}
            max={field.max}
            step={field.step}
            minLength={field.minLength}
            maxLength={field.maxLength}
            pattern={field.pattern}
            autoComplete={field.autocomplete}
            disabled={field.disabled}
            readOnly={field.readonly}
            required={field.required}
          />
        );
    }
  };

  const labelClasses = "block text-sm font-medium text-gray-700 mb-1";
  const errorClasses = "text-red-500 text-xs mt-1";

  return (
    <form onSubmit={handleSubmit} className="space-y-6 w-full max-w-2xl mx-auto p-6 bg-white rounded-lg shadow">
      <div className="space-y-2">
        <h2 className="text-xl font-bold text-gray-900">{formData.heading}</h2>
        <p className="text-sm text-gray-500">{formData.description}</p>
      </div>

      <div className="space-y-4">
        {formData.fields.map((field, index) => (
          <div key={index} className="space-y-1">
            <label className={labelClasses}>{field.label}</label>
            {renderField(field)}
            {field.description && (
              <p className="text-sm text-gray-500 mt-1">{field.description}</p>
            )}
          </div>
        ))}
      </div>

      <div className="flex justify-end">
        <button
          type="submit"
          className="px-4 py-2 bg-blue-600 text-white rounded-md hover:bg-blue-700 focus:outline-none focus:ring-2 focus:ring-blue-500 focus:ring-offset-2"
        >
          {formData.submitButton?.label || 'Submit'}
        </button>
      </div>
    </form>
  );
};

export default FormInput;

4. Submitting Form Response

After collecting user input, submit it back to the task:

const submitFormResponse = async (taskId, formData) => {
  const response = await fetch(
    `https://api.browserable.ai/api/v1/task/${taskId}/run/tool-input`,
    {
      method: "POST",
      headers: {
        "x-api-key": "YOUR_API_KEY",
        "Content-Type": "application/json",
      },
      body: JSON.stringify({
        input: JSON.stringify(formData),
        toolCallId: "TOOL_CALL_ID", // Get this from the task status
      }),
    }
  );
  return await response.json();
};

Supported Form Field Types

The form tool we have shared above, supports various field types:

text: Standard text input
password: Masked password input
email: Email input with validation
number: Numeric input
checkbox: Single checkbox
radio: Radio button group
select: Dropdown selection
textarea: Multi-line text input
switch: Toggle switch
range: Range slider

Each field type can be customized with various properties like placeholder, required, min, max, etc.

Stay Updated

Join our Discord community to:

Share your tool ideas and requirements
Ask any queries to get support from Browserable team

Getting Started

Development

Guides

Customization

Admin Interface

Custom Functions & Tools