Since the start of electronic computation, one of the main uses of computers has been for scientific numerical computation. Soon after, applications to management began to control the scene, as far as the volume of computation assigned to them was concerned. Nevertheless, scientific applications are still the most prestigious, especially if we look at the performance required of the computer: the most powerful are usually reserved for scientific computation. Very often, applications lead to mathematical problems which in their complete form cannot be easily solved with exact formulas. One often restricts oneself then to special cases or simplified models which can be precisely analyzed. In most cases, one therefore reduces the problem to a linear one, for example, a boundary value problem of a linear ordinary or partial differential equation. Such an approach can be very effective, and quite often leads to concepts and points of view which can at least qualitatively be used even in the unreduced problems. Nowadays, our aim is to develop iterative algorithms and mathematical analysis for solving large linear systems which come from enormously complicated three dimensional problems using high performance parallel computers. Our research contains a wide spectrum of topics related to the GMRES algorithm such as searching for an adaptive technique of restart frequency, developing a hierarchical basis of preconditioning technique, utilizing software research tools of scientific visualization for a modern high performance parallel computer.