There are potential impacts of Potentially Toxic Elements (PTEs) (e.g., Cd, Cr, Ni, Cu, As, Zn, Hg, and Pb) in soil from the perspective of the ecological environment and human health, and assessing the pollution and risk level of soil will play an important role in formulating policies for soil pollution control. Lingyuan, in the west of Liaoning Province, China, is a typical low-relief terrain of a hilly area. The object of study in this research is the topsoil of farmland in this area, of which 71 soil samples are collected. In this study, research methods, such as the Nemerow Index, Potential Ecological Hazard Index, Ecological Risk Quotient, Environmental Exposure Hazard Analysis, Positive Matrix Factorization Model, and Land Statistical Analysis, are used for systematical assessment of the pollution scale, pollution level, and source of PTEs, as well as the ecological environmental risks and health risks in the study area. The main conclusions are: The average contents of As, Cd, Cr, Cu, Hg, Zn, Ni, and Pb of the soil are 5.32 mg/kg, 0.31 mg/kg, 50.44 mg/kg, 47.05 mg/kg, 0.03 mg/kg, 79.36 mg/kg, 26.01 mg/kg, and 35.65 mg/kg, respectively. The contents of Cd, Cu, Zn, and Pb exceed the background value of local soil; Cd content of some study plots exceeds the National Soil Environmental Quality Standard Value (0.6 mg/kg), and the exceeding standard rate of study plots is 5.63%; the comprehensive potential ecological hazard assessment in the study area indicates that the PTEs are at a slight ecological risk; probabilistic hazard quotient assessment indicates that the influence of PTEs on species caused by Cu is at a slight level (p
= 10.93%), and Zn, Pb, and Cd are at an acceptable level. For the ecological process, Zn is at a medium level (p
= 25.78%), Cu is at a slight level (19.77%), and the influence of Cd and Pb are acceptable; human health hazard assessment states that the Non-carcinogenic comprehensive health hazard index HI = 0.16 < 1, indicating that PTEs in soil have no significant effect on people’s health through exposure; the PMF model (Positive Matrix Factorization) shows that the contribution rates of agricultural source, industrial source, atmospheric dust source, and natural source are 13.15%, 25.33%, 18.47%, and 43.05%, respectively.