Yes, physics is much harder than English. Physics involves a lot of mathematics and problem solving, while English is mainly about grammar and understanding the nuances of language.