Kardelen Cihangir

Let’s dive into the latest UiPath update to explore the advanced features of Computer Vision and uncover a variety of tools to effectively navigate and manage complex user interfaces.

🔧 Activities:

🖱️ CV Dropdown Select:
Utilize the power of Computer Vision to seamlessly select items from dropdown menus. Specify your choice through Text or Index properties for effortless automation.

☑️ CV Check:
Toggle checkboxes or radio buttons with ease. Utilize the Computer Vision engine to identify elements and the Action property to check or uncheck with precision.

📊 CV Extract Table:
Extract tabular data directly from your screen. The Computer Vision engine carefully identifies tables, rows, and columns, while ExtractOptions allow for tailored data extraction and storage.

🖱️ CV Click:
Execute clicks on screen elements identified by the Computer Vision engine. Choose your click type—single, double, or long—for dynamic interaction.

🔍 CV Screen Scope:
A foundational activity that initializes the Computer Vision engine and sets a domain for element selection within your screen. Adjust the Accuracy property to refine element identification.

✨ CV Highlight:
Visually spotlight elements on your screen, with customization options for color and duration, enhancing visibility and focus.

🖱️ CV Hover:
Navigate your cursor over screen elements with precision. Adjust the delay post-hover to optimize process flows.

✍️ CV Type Into:
Input text into identified fields with accuracy. Specify text and whether to clear the field beforehand, ensuring data integrity.

📝 CV Get Text:
Retrieve text from screen elements for processing or validation, storing results efficiently in a variable.

❓ CV Element Exists:
Verify the presence of screen elements, storing outcomes in a Boolean variable for conditional logic implementation.

The activities mentioned above can be configured at design time by using the Indicate On Screen button in the body of the activities. Clicking the Indicate On Screen (hotkey: I) button opens the helper wizard. The wizard has the following features:

👉 Indicate
The Indicate field specifies what you are indicating at the moment. When the helper is opened for the first time, the Target needs to be indicated. For each possible target, the wizard automatically selects an anchor, if one is available.

👁️ Show elements
The Show Elements (hotkey: s) button in the wizard highlights all UI elements that have been identified by the Computer Vision analysis.

🔄 Screen Refresh
The Screen Refresh (hotkey: F5) button can be used at design time, in case something changes in the target app, enabling you to send a new picture to the Computer Vision server to be analyzed again.

⏳ Delayed Screen Refresh
The Delayed Screen Refresh (hotkey: F2) button performs a refresh of the target app after waiting 3 seconds. The run-time auto-scroll direction is also captured (ScrollDirection property).

🎯 Relative To
The activities that have the Indicate wizard also include a Relative To feature, where the target is a stable element and you drag a distance relative to it. The main target, the stable one, acts as a proxy target for the second target, the relative one.
This is useful when you want to select an area that is not correspondent to the UI elements that the Computer Vision engine has identified, or when you want to select a general area in the UI without specifying a certain UI element.
The relative target can be either a single point or an area selection in the application you are automating.
Single point selection is normally placed in a spot where Computer Vision detection is not accurate and might not find certain elements. The activities that use single point selection are CV Click, CV Hover, and CV Type Into.

📊 Table Indication
The Computer Vision activities also offer support for indicating tables. Targeting in tables can be done by selecting a cell you want to interact with, which prompts the neural network to automatically identify the column and the row that define the position of that cell, displaying them in a grid.
By default, the names of the column and row are used in the descriptor to pinpoint the location of the cell. Clicking the column and row indexes automatically adds them to your descriptor. This might be useful in situations where column and row names are changed, but you want to extract the same position of a cell.

That’s it for the guide on using the Computer Vision activities in UiPath Studio.

Happy automating!