You will need to create a CSV. Then you can can just copy and paste the first column into the text layer column.
The best and easiest way to create a CSV or TSV is by using the CSV/TSV starter in SPA. This will save time by auto populating the source image filenames in Column A. It will also ensure that the filenames and headers are all filled out correctly. This will help avoid issues when processing.
In you case, after creating the CSV using the CSV starter, just open in excel and copy all of the filename and paste under the text layer header.
Doing it this way, you won’t have to manually type anything.