IMAP email not being received - webklex-imap

[glennmuk]

Bug report

Title

IMAP email not being received - webklex-imap

Issue Description

NO IMAP EMAIL RECEIVED
After setting up my IMAP settings in both the .env file and also Configuration --> IMAP Settings, I cannot receive any email.
Note: I can send email OK via smtp.

Preconditions

Please provide as detailed information about your environment as possible.

1. KrayinCRM - Version 2.1.2.

Steps to reproduce

It is important to provide a set of clear steps to reproduce this bug.If relevant please include code samples.

In the .env file - I have set:
MAIL_RECEIVER_DRIVER=webklex-imap
IMAP_HOST=mail.hostname.com (example)
IMAP_PORT=993
IMAP_ENCRYPTION=ssl
IMAP_VALIDATE_CERT=true
IMAP_USERNAME=[email protected] (example)
IMAP_PASSWORD=myEmailPassword (example)

Also, in my application via my admin login - Configuration --> IMAP Settings in the application, I have also set the following:
Host - mail.hostname.com (example)
Port - 993
Encryption Type - ssl
Validate SSL Certificate - true
IMAP Username - [email protected] (example)
IMAP Password - myEmailPassword (example)

What other settings should I be updating?

I cannot receive any email.

Expected result

Email should be received.

• [Screenshots, logs or description]

Actual result

No email received

• [points....]

[sagarkumar-webkul]

Hello @glennmuk ,
Thanks for raising this issue! You can check out our blog or configuration guide for detailed steps. If you still run into any problems, feel free to reach out — we’re happy to help.
blog link - https://krayincrm.com/how-to-configure-imap-settings-in-krayin-crm/

[pekiZG]

Dropping this here and if needed I can provide more info.

Had the same problem, switched to pop3 protocol and different port 995 (pop3 is ussualy on this port)
And mail started to appear. When you do that you have another problem.
If incoming email has attachement then you get this error:

Processing the incoming emails.

In AttachmentRepository.php line 50:

  Webkul\Email\Repositories\AttachmentRepository::prepareData(): Argument #2 ($attachment) must be of type Illuminate\Http\UploadedFile, Webklex\PHPIMAP\Attachment given, called in /var/www/cms-partus-akademija.defaultsolutions.io/htm
  l/packages/Webkul/Email/src/Repositories/AttachmentRepository.php on line 34

I did not have time to test attachements more (to see if it's breaking on two attachements, or some extensions)

@sagarkumar-webkul if needed I can provide more info.

EDIT:
If you invoke artisan inbound-emails:process second time then it passes, but you dont see the attachements in inbox.

---

laravel-crm/packages/Webkul/Email/src/Repositories/AttachmentRepository.php

Line 50 in ca2b66e

private function prepareData(Email $email, UploadedFile $attachment): array

[sagarkumar-webkul]

Hello @pekiZG,
Thank you for providing more details. I was able to reproduce the issue during testing, and we will work on fixing it soon. Appreciate your help in reporting this!

vidoe link - https://webkul.chatwhizz.com/share/view-recording/68a463f0b7af9f05d399dd8c

[pekiZG]

Nice!

But that is tied to pop3 protocol settigns wich I described.
What about IMAP? Did you manage to reproduce that part?

[sagarkumar-webkul]

I have tried with the Gmail IMAP configuration, which is:

Host: imap.gmail.com

Port: 993

Encryption: SSL/TLS

[pekiZG]

Tnx;
I have IMAP from Hetzner (EU provider from Germany).
And it's not working form me. And with PPOP3 protocol it's breaking (you managed to reproduce it)

I will debug it (IMAP settings) and write down what I have found in my case.
Do I open new Issue or just write here?

[sagarkumar-webkul]

You can mention here

[glennmuk]

Hello @glennmuk , Thanks for raising this issue! You can check out our blog or configuration guide for detailed steps. If you still run into any problems, feel free to reach out — we’re happy to help. blog link - https://krayincrm.com/how-to-configure-imap-settings-in-krayin-crm/

Sorry for the delay. I am only getting back to this now.

The instructions in the User Docs and Dev Docs didn't mention the need to run the command. php artisan inbound-email:process

Thanks for the blog link. I have followed the steps in the blog but receive this sendgrid error. I didn't expect this error as I have updated my .env file to webklex-imap, so it shouldn't be looking for sendgrid.
My .env file has this: MAIL_RECEIVER_DRIVER=webklex-imap

Error received when trying to run 'php artisan inbound-email:process'

(
In SendgridEmailProcessor.php line 35:
Currently bulk processing is not supported for Sendgrid.
)

Do I need to update something else to bypass sendgrid? It should really be trying to use this file: WebklexImapEmailProcessor.php

[suraj-webkul]

Hello, @glennmuk

If the MAIL_RECEIVER_DRIVER is set to webklex-imap, then the application should be creating an instance of WebklexImapEmailProcessor for interacting with emails. Since you are still seeing the error coming from SendgridEmailProcessor, it would be good to double-check that your .env file is updated correctly and that the config cache isn’t serving an old value.

You can try running:

php artisan config:clear
php artisan cache:clear

to make sure the latest .env values are being picked up. After that, re-run the command and see if the issue persists.

[glennmuk]

OK, I ran these two commands and it seems to have worked this time. I no longer get the sendgrid error.

However, I still receive no email.

I have just noticed something interesting when sending email. My email is sent. When the person tries to reply to my email, my email address changes to a UNIX epoch number (timestamp). For example: [email protected]
If they try to send an email to this address, it is not received. They get a MAILER-DAEMON email notification.

Any ideas?

[pekiZG]

@glennmuk I just commented out this line:

laravel-crm/packages/Webkul/Email/src/Mails/Email.php

Line 30 in ca2b66e

->replyTo($this->email->parent_id ? $this->email->parent->unique_id : $this->email->unique_id)

But I'm not sure if it was right thing to do.
From my understanding that is unique identificator for something.
Not sure for what? Could be tied to threading email conversations?

Maybe @sagarkumar-webkul, @suraj-webkul or someone else knows more :)

[suraj-webkul]

Hello, @pekiZG, @glennmuk

You’re right — the replyTo line is not just a random field, it’s used for threading email conversations inside the CRM. The system generates a [email protected] for each email so that when someone replies, the CRM can capture that reply and link it back to the correct parent message or conversation.

If you comment out that line, the email will still go through and replies will come back to the from address, but the CRM may not be able to automatically associate replies with their original thread. That means we’d lose some of the conversation tracking.

So it depends on the requirement:

• If we only care about sending/receiving emails, commenting it out is fine.
• If we want full conversation threading, we’ll need to keep it and ensure the mail server can handle those unique reply-to addresses.

[pekiZG]

Tnx @suraj-webkul

I commented it out in the beginning until I have set up a catch all for emails on the domain.
I think @glennmuk was just confused why is it like that. And he wants to know how to actually set it up.

EDIT:
Here is explanation on what is catch all or wildcard email address.
https://docs.hetzner.com/konsoleh/account-management/email/mailboxes#the-catchall-account
@glennmuk hope that helps.

[sagarkumar-webkul]

@pekiZG
I have verified the issue where emails with attachments were not being fetched in the inbox. This has now been resolved, and emails with attachments are being fetched successfully.
video link - https://webkul.chatwhizz.com/share/view-recording/68a7f8371bb1b4321f300fe8

[pekiZG]

I see changes done. Give me 15 minutes. I can also confirm on my side.

[pekiZG]

Works also on my side i have testes it with this settigns in .env config.

IMAP_PROTOCOL=pop3
IMAP_HOST=mail.your-server.io
IMAP_PORT=993
IMAP_ENCRYPTION=ssl
IMAP_VALIDATE_CERT=false
IMAP_USERNAME=<REDACT>
IMAP_PASSWORD=<REDACT>

For some reason I can not get it to work with IMAP_PROTOCOL=imap;
Cold be misconfig on my side.

TLDR: attachements work :)
Tnx to all who participated in solving this :)

[pekiZG]

@sagarkumar-webkul it was not a misconfiguration on my side.
@glennmuk I think I figured out why we both did not receive emails on IMAP protocol.

So the thing is that InboundEmailProcessor is traversing through folders (the part where processMessagesFromLeafFolders is being called foreach folder). But there is one return statement that should not be there.
https://github.com/krayin/laravel-crm/blob/2.1/packages/Webkul/Email/src/InboundEmailProcessor/WebklexImapEmailProcessor.php#L167

Why you ask?
Well in my case POP3 worked, because it always returned only INBOX.
IMAP has a hierarchy where we should not skip the parent, so that return statement is redundant and it should not be there.

Solution is to just comment out that return statement.
Can someone confirm this?

Also sidenote for IMAP;
It has default folders like Trash, Junk, Archive, Draft, etc... (I'm sure there is some RFC that describes that)
So either you need to add those names to line 170, or just process INBOX
https://github.com/krayin/laravel-crm/blob/2.1/packages/Webkul/Email/src/InboundEmailProcessor/WebklexImapEmailProcessor.php#L170

My suggestion would be to only process INBOX that would simplify the code.
Reasoning being that if email is not in inbox means that there are some rules that put it there (Example: Junk).
Just my 2 cents :)

EDIT: Ignore the suggestion I just recalled that folders are being mapped, so maybe just process those folders as safeguard?
https://github.com/krayin/laravel-crm/blob/2.1/packages/Webkul/Email/src/InboundEmailProcessor/WebklexImapEmailProcessor.php#L110

[glennmuk]

Thanks @suraj-webkul and @pekiZG

I did some testing on this last night.
My mailserver was not able to handle catch-all email addresses.

There really should be information in the documentation about this CRM email feature.

With further testing, I decided to use a different IMAP host for a different mailbox to see if there might be a difference and yes there was!

Overnight, I have changed my IMAP email provider to this new host and now this is working for me. All other settings remained the same. I was using rackspace email and I could not get my IMAP working at all. My SMTP was OK for sending email.

Now that I have moved my email to my new provider (a local provider in Ireland), my emails are working. And, my new provider allows for the creation of a catch-all email so I can confirm that this is also working. The responses via the [email protected] are threading correctly and the responses are coming into my conversation.

@pekiZG - For your IMAP problem, I would recommend testing it with another email provider.

I don't know why rackspace wasn't working for me but this change has made everything work.

On the attachments, I am still having an issue. I have updated AttachmentRepository.php with new code updates.
I have attempted to send an email to my CRM with 4 attachment files: (1.pdf, 1.sql, 1.docx and 1.png)
When I run the php artisan inbound-email:process, I get this error:

In WebklexImapEmailProcessor.php line 54:
Unable to create a directory at /sitedomain/public_html/laravel-crm/storage/app/public/emails/41.

The email is then received in the CRM, but no attachments are on the email.

Thanks again for all your help.

[pekiZG]

@glennmuk try clearing the cache via
php artisan optimize:clear
and then invoke php artisan inbound-email:process again to see if it works for you.

Regarding my IMAP problem, Im 99% sure that there is logical error in the code.
Tested on one more mail server. With Gmail it works.
POP3 returns only one folder and that is INBOX, IMAP retuns hierarchy so you also need to process the parent.
That can not happen because od the return statement here (https://github.com/krayin/laravel-crm/blob/2.1/packages/Webkul/Email/src/InboundEmailProcessor/WebklexImapEmailProcessor.php#L167).

I can also suggest trying out Cloudflare. You can register domain there, and it has catch all and forward to where you want.
I'm puzzeled with catch all setup, ussusaly you have some identificator in the subject line. It would be nise to have a switch (feature flag) in Settings to either use Threading or not :)

As a last note regarding documentation, you can edit and create PR here.
https://github.com/krayin/krayin-docs
And leave comments and how you set it up :)

[glennmuk]

OK, thanks for your help @suraj-webkul @sagarkumar-webkul @pekiZG .

I have decided to keep my main email out of Krayin CRM and only use a gmail account in Krayin. I won't use the CRM to manage customer emails as there are too many little problems (bugs).

Instead, I will keep my customer emails separate and when I receive an email in that needs to be attached to a lead for records, I will forward it into my CRM (gmail account). That seems to be the best way forward for now.

A couple of things I have noticed.
With the current settings (without editing code in the files), it generates the [email protected]. This is pointless when using gmail as, it creates a number like this: [email protected] when trying to reply to the email. Obviously, this will go nowhere and the CRM won't receive it.
FEATURE REQUEST - You should be able to edit the functionality in the Setting/Configuration UI, to toggle between threading emails or not threading emails (instead of having to comment out lines of code).

On another email, I sent one from my [email protected] to my yahoo account. When I hit reply, it should have tried to reply to [email protected], to go back to my CRM and into the threaded conversation.
But, instead, when I hit reply on this email, if actually tried to email: [email protected]
This is totally random and it will mean that customers cannot reply to me as I will not receive the email. I can't afford to lose emails.

Also, the Inbox is not refreshing at all. I thought it was meant to be every 5 mins as a CRON job? This isn't working for me. I have to keep running this command in the terminal to fetch emails.
php artisan inbound-email:process

FEATURE REQUEST - There should be a button on the Inbox page to Fetch Email.
This button should refresh the mailbox and fetch any new emails. You can still leave the CRON job running every 5 or 10 minutes, if it is working. A button to auto fetch emails while waiting for the CRON job is essential, if you are waiting for an urgent email to come in.
You could place this new button beside the Compose Email button.
Something like this:

Finally, the MAIL_FROM_NAME="${APP_NAME}" from the SMTP settings, does not send. It does not send your app name with the SMTP email. This needs to be fixed.

So for now, I feel the email section is not ready for a general production environment.

If anyone else gets stuck using GMAIL for their Krayin project, I have found that the following settings works for me:

SMTP and IMAP Settings.
I did get my business email working for both smtp and imap but due to the problems above, I decided to use a gmail email account instead. I will keep my business email out of the CRM for now.

GMAIL Settings.
You need to set up App Passwords for Gmail instead of your normal email passwords. It would only work for me if I had two different app passwords. One for SMTP and one for IMAP. When using them in your settings, remove any spaces in the password that is generated by google. (Google gives you 16 character passwords with spaces between four-character blocks, eg: saff werw werg iuoy)

MAIL_MAILER=smtp
MAIL_HOST=smtp.gmail.com
MAIL_PORT=465
MAIL_USERNAME=[email protected]
MAIL_PASSWORD=app_password_1
MAIL_ENCRYPTION=ssl
MAIL_FROM_ADDRESS=[email protected]
MAIL_FROM_NAME="${APP_NAME}"
MAIL_DOMAIN=gmail.com

MAIL_RECEIVER_DRIVER=webklex-imap

IMAP_HOST=imap.gmail.com
IMAP_PORT=993
IMAP_ENCRYPTION=ssl
IMAP_VALIDATE_CERT=true
IMAP_USERNAME=[email protected]
IMAP_PASSWORD=app_password_2

Thanks
Glenn

[pekiZG]

Request for manual fetch button already exists in #2268

[pekiZG]

I see that attachements fix was merged; what about IMAP issue that we are having?
Is more info needed?

[sagarkumar-webkul]

Yes, please share more information

[pekiZG]

Hi, just saw this was moved to discussions.
I have time this weekend; expect update after the weekend;
I will try to be detailed as possible.