Yes. Brain damage (as example) can present a definite effect upon behavior.

Hormones like adrenalin and cortisol are also affecting our behaviour, and also our sex hormones.

This is a very huge question. I will very severely oversimplify my answer. The synapses between Neurons are where behavior 'happens'. The synapses are very complex and utilize hundreds variants of many different neurotransmitters, for instance seratonin or GABA (Gamma-aminobutyric acid), to transmit and inhibit signals. How it all works is dependent on myriad factors. The range is set by the genetic makeup of the organism. This is one Physiological factor as the genes that the organism has specify the particulars of all of the components involved in the system. External factors are the availability if nutrition in the form of glucose and oxygen to facilitate its metabolism into ATP (Adenosine Triphosphate) which provides all of the chemical energy do drive the uphill chemical reactions involved. Signaling hormones alter the characteristics of the system in various ways. Pheromones are also involved in some areas of the brain as well. The time of day effects behavior as our brains contain circadian clocks that regulate brain functions such as hunger and sleep. This is only a very superficial overview of the topic, and is by far not very comprehensive.