The left image is the entrance page window of AMXS.
There are links to various manual pages and reference data at the bottom.
Click the [START] button at the center to start the AMXS system.
| GSJ Open-File Report, no. 655 | August 2018 |
Tadashi Nakatsuka and Shigeo Okuma
(Research Institute of Geology and Geoinformation, GSJ, AIST)
We have been engaged in the research on aeromagnetic survey, and developed various kinds of programs for aeromagnetic data processing, analysis and interpretation. We have already reported the outlines of them on the scientific journals and at scientific meetings [Nakatsuka, 1995; Nakatsuka and Okuma, 2006a, 2006b, 2009a, 2011, 2014a; Nakatsuka et al., 2009b; Okuma et al., 2009c, 2014b; and others]. Their source codes are open to public through the GSJ Open-file Reports.
Recently, as a fully-revised and expanded version for the previous AeroMagnetic Software System (AMSS) [Nakatsuka, 2009d, 2009e], we issued the newest version of it on the GSJ Open-file Report no. 648, "AMSS3: Software system for Aeromagnetic data processing, Grid data manipulation, and Reduction and quantitative interpretation of magnetic anomaly data (3)" [Nakatsuka and Okuma, 2018]. This report also contains the newly developed programs of higher sophisticated analysis methods, such as 3D imaging of subsurface magnetization structure.
This software system is in a style of program libraries, consisting of many individual programs of rather simple functions. Then the actual process for some target/objective would require a series of program executions. In actual execution of each program, the user is prompted for process parameter values including data filenames to be handled, in an interactive style. But this scheme is on the CUI (character user interface), and some user might think it annoying. Then, we developed a workstation environment of GUI (graphic user interface) with user-friendly Web (HTML) interface to perform actual data processing and quantitative interpretation with the full software system AMSS3. We call this environment of workstation "AMXS: AeroMagnetic Expert System". This name does not mean a decision-making system with artificial intelligence, but a convenient mechanism of selecting and practicing a tool out of the abundant tools of various functions, to assist the aeromagnetic experts' data handling and interpretation. There are so many and various kind of tools in the system, it might also be useful for researchers of other fields.
In this report, first we summarize the outline and operating circumstance of AMXS software, next we show the procedure how to build AMXS system and examples of AMXS working window with supplementary notes for the help of effective use of Button utilities, and then the filenames of AMXS source codes are presented in a tree-structure.
AeroMagnetic Expert System (AMXS) is a convenient mechanism of GUI
(graphic user interface) environment to use the Aeromagnetic survey software
system AMSS, which has been developed as a total data processing system
covering from data inspection and quality management to various kind of data
analysis, subsurface structure inversion, and further interpretation.
We adopted the Web (HTML) interface for the GUI, and Web browser's screen
as designed by HTML documents with some Javascript offers the way of
executing any of the AMSS3 programs, by using CGI script of Perl
language.
Here it is noted that our AMXS environment is highly vulnerable
if it is connected to the worldwide network, because we assume the system is
connected to the physically closed network only, and do not provide any
security management.
Accordingly, careful examination will be required if such security
management is required.
The outline of this AMXS system is illustrated in Fig. 1.
The bottom half of the illustration as indicated with "AMSS" means the
executable binaries of data processing, and is completely same as today's
system of AMSS3 programs.
The top half of the illustration is the part of AMXS functions, and it
includes the man-machine interface through the Web browser.
To mention further detail, there are some small utility programs coded in
C language within the part of CGI part, and also the HTML part involves
some PNG image files.
The contents of them are described later as a tree-structure of all
files.
The programs of AMSS3 system are, excluding subprogram library
'libgm' and Miscellaneous utilities 'utils', divided into 4
groups as 'DPAM', 'GDMP', 'ANAM', and 'ANAMX',
the outline of individual functions of each group are summarized in the
Tables 1-4.
For the successful function of this HTML–CGI scheme, the installations of Web server and interpreter of CGI-scripts are essential. Our choice is the Apache Web server and Perl language CGI interpreter on a Linux operating system. Those system service programs are well-known open-source software (free-software), and they are distributed under the Apache license and the GNU public license, respectively.
Table-1 DPAM group programs
|
Table-2 GDMP group programs
| |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
Table-3 ANAM group programs
|
Table-4 ANAMX group programs
|
In this section, we show an example of constructing AMXS system,
the one actually we are operating.
There will be many variations of constructing the system, and such
variations are not discussed here.
In order to construct this system,
(1) Installation of AMSS3 executable binaries, reference
data and manual,
(2) Installation of HTML documents and CGI scripts, and
generating CGI-utility executable binaries,
(3) Configuration of AMXS service (Web service) environment
are the essential steps.
We place the AMSS3 programs in the directory /home/SHARE
of a server machine equipped with Linux OS.
The arrangement of the contents is summarized as follows, with referred to
the tree structure of files in GSJ Open-file Report no.648 [Nakatsuka and
Okuma, 2018] and with wildcard notation.
[Open-file report no.648] [Target Linux machine] [Content]
no0648/*.html /pub/html/man/*.html (Manuals)
no0648/libgm/*.html /pub/html/man/libgm/*.html
no0648/libgm/figs/* /pub/html/man/libgm/figs/*
no0648/lib/* /home/SHARE/lib/* (Source Programs)
no0648/utils/* /home/SHARE/utils/*
no0648/dpam/* /home/SHARE/dpam/*
no0648/gdmp/* /home/SHARE/gdmp/*
no0648/anam/* /home/SHARE/anam/*
no0648/anamx/* /home/SHARE/anamx/*
no0648/Tplate/*/* /home/SHARE/Tplate/*/* (Templates)
no0648/data/ /home/SHARE/data/ (for Service Data)
--- /home/SHARE/bin/ (for Executable binaries)
Here, /home/SHARE/bin is the target directory,
in which executable binaries are generated after compiling the source codes.
After storing source files above, the following command sequence can
complete the installation.
cd /home/SHARE
alias fort gfortran
chmod +x */@mkall
cd lib
./@mkall
cd ../utils
./@mkall
cd ../dpam
./@mkall
cd ../gdmp
./@mkall
cd ../anam
./@mkall
cd ../anamx
./@mkall
With respect to the service data files to be stored into
/home/SHARE/data/, there should be some treatment on the data
as described in GSJ Open-file Report no.648 [Nakatsuka and Okuma, 2018],
and also the user may store additional data files of compatible data
format and filename convention.
On all service data files and manual HTML pages, the user 'apache'
[Web server] should be given the read permission.
The HTML documents, image files, CGI scripts, and some other files included in this report should be stored on the server as follows.
[This report] [Target server machine] [Content]
no0655/html/AMXS/* /pub/html/AMXS/* (HTML documents)
no0655/html/AMXS/forms/* /pub/html/AMXS/forms/*
no0655/html/AMXS/progs/* /pub/html/AMXS/progs/*
no0655/html/AMXS/fout/ /pub/html/AMXS/fout/ (Work Area)
no0655/cgi/AMXS/* /pub/cgi/AMXS/* (CGI scripts)
no0655/cgi/AMXS/bin/ /pub/cgi/AMXS/bin/ (for CGI-Utility binary)
no0655/cgi/AMXS/bin/src/* /pub/cgi/AMXS/bin/src/* (CGI-Utility source)
The source files of small utilities are stored in the directory
/pub/cgi/AMXS/bin/src.
These programs are compiled with the shell script @mkall in
the same directory, and the executable binaries are generated in the parent
directory /pub/cgi/AMXS/bin, although
'zlib' and 'libpng' libraries must
be implemented prior to the compilation.
It is necessary that the locations of installed HTML documents and
CGI scripts are notified to the Web server (Apache).
Then the Apache configuration file /etc/httpd/conf/httpd.conf
should be edited to contain the directives,
DocumentRoot "/pub/html"
ScriptAlias /cgi/ "/pub/cgi/"
and various settings on these directories should be proper
for the Web service.
On the HTML documents installed, the user 'apache' [Web server]
should be given the read permission, and on the CGI scripts installed
(having the filename extension '.cgi') and CGI-Utility executable binaries,
the user 'apache' [Web server] should be given the read and execute
permission.
Generally, the executable binary files generated by compilers are set on
the execute-permission flag, but the CGI scripts only stored into
corresponding directory are not.
So, it will be necessary to set them by
"chmod +x *.cgi" command execution.
The data handled in actual AMXS execution are stored under
the parent directory /pub/AMXS/ with single layer subdirectory
structure.
This parent directory should be set the access permission so that the user
'apache' [Web server] can create/read/write/delete directories and files.
Likewise, the directory /pub/html/AMXS/fout/ is used
for internal work area, which should be set the access permission for
the user 'apache' to create/read/write/delete files.
The AMSS3 software was issued in January this year, and about half a year has passed. During this time, some possible updates of the programs were recognized. The situation of the existing program is not so bad, but log file outputs from several programs are not complete, and the screen output information from loop operation programs (using conjugate gradient method) is not timely enough (delayed by buffering) only for the case of running under AMXS environment. The last point means that the actual progress of program execution cannot be monitored under the AMXS environment.
The contents of "recommended updates on AMSS3" are in the
AMSS3patch directory (this report), as 4 patch files and 4 plain
text files of update-memo.
These patch files are in the style of "directory diff" for each source code
directory, and the number of source files actually updated is 35.
To apply these patches, first store the patch files into
/home/SHARE/@patch directory, and then the following command
sequence will be useful.
cd /home/SHARE/
mkdir ORG
cp -Rp lib utils dpam gdmp anam anamx Tplate ORG
patch -p0 < @patch/anam_201807.patch
patch -p0 < @patch/anamx_201807.patch
patch -p0 < @patch/dpam_201807.patch
patch -p0 < @patch/dpam_tp_201807.patch
The left image is the entrance page window of AMXS.
There are links to various manual pages and reference data at the bottom.
Click the [START] button at the center to start the AMXS system.
The left image is the top page window of AMXS consisting of 4 frames.
Among 4 frames, two frames ('Frun' and 'Fhid' frames) are usually hidden
because they are only for internal use.
The bottom part of the image (blank space initially) is the 'Fbot'
frame.
When any action or operation is taken, this frame is used to select
the detail parameter, or to show the result of the action/operation.
In the 'Ftop' frame as in the left image, there is a part of setting
working environment and files manipulation on the left, and to the right
there are lists of executable programs.
Blue triangles with "−" or "+" notation are used to fold or unfold
the list content in the group by clicking it.
Each label of the list is the name of program, and itself is a link to
the program's page of setting-up parameters.
The working directory should be assigned to each project or
a sequence of program executions.
As the setup of working environment, the working directory and the log
filename are defined at the left.
Also, the functions of uploading data files, reviewing the text file of
execution result, or previewing the graphic (PostScript) output files
with the help of "GSview" software are available.
With regard to the buttons in the screen, [Select] or [CreateNew]
buttons selects or newly creates directory or file, [Files] button lists up
the filenames in the directory, the button [ls -l] displays like a Unix
command "ls -l", and [tail] button displays about 10 lines to the end of
log file.
Underlined blue strings "Download", "Show", or "Remove" executes its
action to the file assigned in the Input box.
[GSview] button is used to select a file from the list of graphic
(PostScript) files in the directory and preview the graphic content with
the help of "GSview" software installed in the client machine.
[FileUpload] button is used to upload any data file from the client PC.
The images on the left are the samples from actual window of
parameters settling for individual programs execution.
The upper shows the [Data-Range] button action in the "plmapc" program,
and the lower is the [cal IGRF] button action in the "eimga" program.
In each window, the Program name and its brief explanation is shown
at the top center, and on its both wing are [Back] button to go back to
top page of program selection and [Clear] button to reset the parameter
settings.
And below the area of parameters setting shown as bluish background color,
there is a [GO] button to command the program execution.
To define parameter values, "Input box" is used commonly to accept
wide possible parameters.
Also, "Select box" / "Radio button" to select from limited number of
choices, and "Check box" to mean the selection from ON/OFF or Yes/No are
utilized.
And the buttons on the screen are implemented to act in various convenient
style, such as (1) showing in Fbot frame a useful information of setting
parameter, (2) selecting from the list of filenames (in the Fbot frame)
to fill the corresponding filename field, and (3) others.
The upper image of [Data-Range] button action reveals that the
grid parameter and the range of data are shown to assist the contouring
parameter setting, and the lower image of [cal IGRF] button offer a IGRF
calculator to inform IGRF field direction for selecting the direction
parameters.
The functions of common buttons [Select], [Files], [ls -l], [tail -n], and [head -n] are same as those in the top page of program selection.
As seen in the previous subsection, various mechanisms are implemented in the window of parameter setting for individual program. Among them, those of rather simple functions will not require further explanation, but there exist several utilities of particular function in AMXS system. The functions of the [Data-Range] button, and the [cal IGRF] button are mentioned in the previous subsection, and they are implemented to 17 programs, and 13 programs, respectively. Here, we deal with other items below.
The AMXS system often deal with the 2-dimensional distribution of
physical quantities as a grid data, and there are many cases of limiting the
range of available data.
To define the range parameters, we developed utility programs to fix
parameter values by the mouse operation on some graphic representation
of source data, and implemented it to 14 programs as activated by clicking
[Graphic] button.
There are two variations of utility to operate on the color-graded shading
map of grid data, and on the monochrome trackline map of survey lines, and
both are activated with [Graphic] button click.
The above image is an example of [Graphic] button utility, to get
new grid data from existing magnetic anomaly database.
After specifying input data file, the click of [Graphic] button activates
the utility process as in the child window on the right.
The mouse operation can select and adjust the rectangular range.
Then, if you click [OK] button on the child window, the information of the
selected range is transferred to parent window (in the blue ellipse part),
and the child window is closed.
In the image above, there is a [Read from Org.] button. This button is used to take in the range information of input grid data as it is, and it is useful to set range information to a correct relative position by changing the values by hand. This kind of button utilities with the name like [Read from ...] are implemented in 9 programs.
In the window of 9 programs, there is a button [Proj. Inf.] to show a illustration of various map projections implemented. Also, in the window of 4 programs, there is a button [IGRF Inf.] to show a illustration of the history of IGRF generation number referred to its valid year range.
When it takes more than a few seconds to complete the program
execution, the window screen varies from the start to the completion of
program execution like the illustration above.
The Ftop frame displays the last 20 lines of the character output from
the program, and the Fbot frame displays the message "running" with yellow
background, before the completion of program execution.
During the program execution, there is a [ABORT execution] button, and
clicking this button commands the enforced termination of execution.
When the execution is completed, the background color of Fbot frame
is changed to aqua, with a message "completed", and all the character
output from the program are displayed on the Ftop frame (mostly scrolling is
required to view the whole of it), and at the bottom of Ftop frame is a [OK]
button, as is described in the Fbot frame with aqua background color.
To proceed from this completion window, it is required to click the [OK]
button at the bottom of Ftop frame, we should confirm the normal completion
of the program execution.
If the program stopped with error unfortunately, the content of
the error status should be shown just before the [OK] button, then be sure
to read carefully the error status.
This error information disappears after clicking the [OK] button,
while no messages are recorded in the log file on error completion.
This is a specification of AMXS, so be careful!
At the start of program execution, AMXS checks simply for the
validity of parameter values, and if any invalid parameter is detected,
the warning message like the right image is shown with orange background
color in Fbot frame, and the start of execution is suppressed.
After confirming the normal completion of program execution,
the window proceeds to the top page of program selection.
To examine the execution result, graphic (PostScript) output file may
be previewed using "GSview" software, log file or other text file may be
displayed, or arbitrary file may be downloaded to the client machine.
Here, the software "GSview" (or the equivalent) is necessary to be
installed in the client machine, together with the Ghostscript software.
In the illustration above, the left image is the example of
previewing the graphic (PostScript) file selected from .ps files list,
and the right image is the example of displaying a log file (selected
simple text file).
Likewise, by selecting arbitrary file and then clicking the underlined
blue string "Download", the selected file can be downloaded to save it into
the client PC.
In the CD-ROM appended to this report, HTML files, CGI perl script files, and some other files necessary to build the AMXS workstation are stored together with the electronic files of this documentation. The tree structure of them is as follows.
openfile0655.html (Cover page HTML)
no0655/ (Directory for all contents except Cover page HTML)
|
+-- indexE.html (Overview of this report: This document)
+-- index.html (Overview of this report in Japanese)
+-- fig1.png, fig5-1.png, fig5-2.png, fig5-3.png,
| fig5-4.png, fig5-5.png, fig5-6.png, fig5-7.png
| (8 PNG images))
+-- html.tgz (TGZ archive of whole 'html' directory)
+-- cgi.tgz (TGZ archive of whole 'cgi' directory)
+-- AMSS3patch.tgz (TGZ archive of whole 'AMSS3patch' directory)
|
+-- html/
| |
| +-- AMXS/ (Directory for contents of HTML pages of AMXS)
| |
| +-- index.html (Top page jumping to 'entry.html')
| +-- entry.html (AMXS entrance page)
| +-- amxs.css (Common style-sheet definition)
| +-- gsj.png, aist.png (2 PNG images)
| |
| +-- forms/ (Directory for HTML to select program)
| | |
| | +-- axmain.html, axhid.html, axtop.html,
| | | axtop.0.html, axtop.1.html, axtop.2.html, axtop.3.html,
| | | axtop.4.html, axtop.5.html, axtop.6.html, axtop.7.html,
| | | axtop.8.html, axtop.9.html, nothing.html, blank.html,
| | | selcda.html, selcdc.html, igrf.html, proj.html
| | | (19 HTML files)
| | +-- arrow.png, show.png, hide.png, igrf.png, proj.png
| | | (5 PNG images)
| | +-- apsetgp.js (1 Javascript file)
| |
| +-- progs/ (Directory for individual program HTML)
| | |
| | +-- aaptdp.html, adeqc.html, adeqsc.html, adjlv.html,
| | aimgnc.html, aimgsc.html, alog2asc.html, altchg.html,
| | altx.html, amagc.html, ameqc.html, ameqsc.html,
| | axdeqc.html, bdaq2asc.html, calmas.html, cdeq.html,
| | cdeqs.html, cimgnc.html, cimgsc.html, cmag.html,
| | cmagf.html, cmeq.html, cmeqs.html, cxdeq.html,
| | daq2asc.html, despike.html, dmaq2asc.html, dvcorr.html,
| | ecomp.html, edeq.html, edeqs.html, eimga.html,
| | eimgd.html, emag.html, emagf.html, emeq.html,
| | emeqs.html, exdeq123.html, exdeq4.html, fcomp.html,
| | filtadasc.html, fimgsc.html, gadd.html, galtf.html,
| | galts.html, genroff.html, ggrid.html, ggrids.html,
| | gmerge.html, gojoin.html, govlay.html, gsub.html,
| | gtopo.html, gtrf.html, gtrim.html, lcecorr.html,
| | nimgnc.html, nimgsc.html, pchkcomp.html, pchkdv.html,
| | pchkmag.html, pchkres.html, pframe.html, plamag.html,
| | plamagc.html, plimv.html, plimvc.html, plmap.html,
| | plmapc.html, plmapcl.html, plmapcs.html, plmapg.html,
| | plmapl.html, plmaps.html, plmvarc.html, plsim.html,
| | plsimc.html, pltrk.html, plxim.html, plximc.html,
| | rearx.html, rpdeqs.html, rpmeqs.html, sel.html,
| | seldb.html, seldb2.html, shade.html, tmcfix.html,
| | tmcorr.html, txproj.html, xldam.html, xldhg.html,
| | xldpn.html, xplmap.html, xplmapc.html, xplmapcs.html,
| | xslin.html, xslina.html
| | (98 HTML files)
| |
| +-- fout/ (Internal work area)
| |
| +-- @memo.txt (Information on 'fout' subdirectory)
|
+-- cgi/
| |
| +-- AMXS/ (Directory for contents of CGI scripts
| | and CGI utilities called from CGI)
| +-- acqPATHs.pl (Common setting perl script)
| +-- caligrf.cgi, complete.cgi, defgp.cgi, defgpdb.cgi,
| | defpgp.cgi, dnload.cgi, dnload2.cgi, drange.cgi,
| | exabort.cgi, files.cgi, gsview.cgi, head.cgi,
| | headdb.cgi, headg.cgi, lslong.cgi, newlog.cgi,
| | newlog2.cgi, newwdr.cgi, newwdr2.cgi, psdload.cgi,
| | refresh.cgi, refresh1.cgi, remove.cgi, remove2.cgi,
| | rmcwd.cgi, rmcwd2.cgi, runprog.cgi, runprog2.cgi,
| | runprogx.cgi, selfl.cgi, selfnm.cgi, sellog.cgi,
| | selwdr.cgi, setgp.cgi, setgp1.cgi, setgpdb.cgi,
| | setgpr.cgi, show.cgi, tail.cgi, upload.cgi,
| | upload2.cgi (41 CGI perl scripts)
| +-- setgp.par (Parameter data for mkmpng/mktpng utility)
| |
| +-- bin/ (Directory for CGI utility binaries)
| |
| +-- src/ (Directory for CGI utility sources)
| |
| +-- @mkall (Script to build CGIutil binaries)
| +-- darea.c, drange.c, igrfidf.c, mkmpng.c, mktpng.c
| | (C language sources of CGIutil)
| +-- setgp_template.png
| (PNG image only for reference)
|
+-- AMSS3patch/ (Directory for patch files against AMSS3 sources)
| |
| +-- dpam_201807.patch, anam_201807.patch, anamx_201807.patch
| | (3 patch files for source program directories)
| +-- dpam_tp_201807.patch (1 patch file for Tplate/dpam_tp directory)
| +-- @update-dpam.txt, @update-anam.txt, @update-anamx.txt,
| @updateTP-dpam_tp.txt (4 text files of notes on source update)
|
+-- PDFs/ (Directory for PDF files of this report)
|
+-- CoverPage.pdf (PDF version of this report - Cover page)
+-- English.pdf (PDF version of this report - English part)
+-- Japanese.pdf (PDF version of this report - Japanese part)
References
[in chronological order]