pyautogui locateCenterOnScreen is not finding my source image or 'needle'

[renegadeandy]

I am searching in an area of 543 pixels wide and 378 pixels height from the top left corner of my screen for the following 'needle'

You may recognise the Firefox logo.

My search area looks like this:

I am using the following code:

import pyautogui
import time

search_region = (0,0,543,378)
x, y = pyautogui.locateCenterOnScreen('graphics/firefox_icon.png',grayscale=True,confidence = 0.4,region=search_region)
print("We found it at X:"+str(x)+" Y:"+str(y))
pyautogui.click(x, y)

When executing the code using a confidence value of 1,0.9,0.8,0.7,0.6,0.5 it raises an ImageNotFoundException. When using confidence of 0.4 it usually selects a pixel just under the finder menu which is in the clouds(coords X:80, Y:69). The same happens for grayscale=True or False. This is my challenge, I cannot get a simple use case like this to work - and I am not sure what I am doing wrong. Please offer some options I can try to fix my code or my configuration. As a note - I am using Macs 'Shift - Cmd - 4' short cut to select the area to create my needle image - which is output as a png by default, incase that is perhaps related.

My environment:

Python version: 3.12.4
pyautolib version: 0.9.54
opencv-python version: 4.10.0.84
Mac OS version: 14.5
MacBook Pro M1(retina display) in dual monitor mode with a 34" widescreen

[clcaod]

I have the same issue, My M1 Mac has never successfully used this function. If I reduce the confidence to 0.5, I can find it, but the position is not correct, which is very troublesome

There is a problem here. I obtained an image size of 1880*2800 by calling cv2. screenshot(), but my resolution is 1440 * 900. I am not sure if this has any impact

[clcaod]

@renegadeandy

I have solved my problem. The reason is that the resolution of the bottom screenshot is doubled. We need to adjust the size after screenshots, and then match the template.

As for why the screenshot size is doubled, you can refer to this issue

def locate_img():    
    target_img = cv2.imread('obsidian.png', cv2.IMREAD_GRAYSCALE)
    # 截屏默认为RGBA
    screen = ImageGrab.grab()
    # numpy将图片转为数组数据
    screen_tmp = np.array(screen)
    # 将RGBA通道转为灰度图
    screen_img = cv2.cvtColor(screen_tmp, cv2.COLOR_RGBA2GRAY)
    print("screenshot_size：" + str(screen_img.shape))
    # 调整截屏大小为屏幕分辨率大小
    width, height = pyautogui.size()
    screen_img = cv2.resize(screen_img, (width, height))
    print("screenshot_resize：" + str(screen_img.shape))

    result = cv2.matchTemplate(screen_img, target_img, cv2.TM_CCOEFF_NORMED)
    min_val, max_val, min_loc, max_loc = cv2.minMaxLoc(result)
    print(min_val, max_val, min_loc, max_loc)

output

screenshot_size：(1800, 2880)
screenshot_resize：(900, 1440)
-0.6599576473236084 0.901120662689209 (340, 846) (874, 846)