Try these steps to see if the master is running and accepting connections:
1) From the same EC2 instance where you ran start-master.sh, find the master log file. When you start the master, it should report where it is logging. You can use stop-master.sh then start-master.sh again to see this info. For my master, I see:
starting org.apache.spark.deploy.master.Master, logging to /opt/spark/logs/spark-root-org.apache.spark.deploy.master.Master-1-ip-172-31-24-101.out
2) Tail this log in a terminal window.
tail -f <thePathToTheLogFile>
The last thing I see in the log before doing anything useful is:
24/10/22 10:16:58 INFO Master: I have been elected leader! New state: ALIVE
3) In a 2nd terminal window on the same EC2 instance, try starting up an interactive Spark shell pointing at the master. For my master IP, I run.
$SPARK_HOME/bin/pyspark --master spark://ip-172-31-24-101:7077
4) Observe the log and confirm that the master accepted the connection from the interactive shell. In my log, I see:
24/10/22 10:17:42 INFO Master: Registering app PySparkShell
24/10/22 10:17:42 INFO Master: Registered app PySparkShell with ID app-20241022101742-0000
If all 4 steps worked, your master is running, so the issue might be somewhere between the master and wherever you're running spark-submit.sh from.
Regards,
BU