I found the solution from LILYGO Support itself.
For anyone wondering how to solve this, you need to hold the RST pin of LoRa high before ESP32 goes to sleep so that LoRa will be active.
Add this line before going to sleep
gpio_hold_en(GPIO_NUM_14);
The full sleep function will look like this.
void sleep_controller(long seconds) { //Deep Sleep ESP32
Serial.println("Sleeping for " + String(seconds) + " secs");
esp_sleep_enable_timer_wakeup(1000000LL * seconds);
// Hold the RST pin of LoRa before going to sleep
gpio_hold_en(GPIO_NUM_14);
esp_sleep_enable_ext0_wakeup(LORA_DIO0_INT, 1);
esp_deep_sleep_start();
}