User Experience (UX) design aims to create intuitive, ef- ficient and enjoyable digital services. User satisfaction is an important factor in the success or failure of a digital product or service. A literature review by Lu et al. on human-centered AI shows that popular areas of AI use for UX are review mining (e.g. analysing user reviews), persona creation or AI-assisted brainstorming. Various aspects of UX and usability testing, such as accessibility and responsiveness, have also been automated. However, some parts still require human participants, which makes it expensive and time consuming. With the advancement of AI and its growing ability to replicate human actions, the potential to use it specifically for the purpose of us- ability testing is emerging. In particular, Large Language Models (LLM) offer the possibility of chatting with an AI that aims to replicate chatting with a human, which comes close to the interaction during a usability test with humans.