Page Index:
HOME /
Install Huge Catalogs /
Install Small Catalogs /
Information about table columns /
Hard Drive Copy Service
Before installing catalogs, you have to setup OS, PostgreSQL and 2MASS Kit following the installation procedure.
We show the procedure of installing 2MASS PSC.
You can also install other huge catalogs supported in this kit
with instructions in this page.
Replace `twomass
' when installing other catalogs.
Download all required data files and then extract them all.
Here is an example using wget command:
$ mkdir twomass_data $ cd twomass_data $ wget -nd -r http://darts.jaxa.jp/pub/ir/2masskit/v2/twomass/ $ gunzip *.db.txt.gz
In `twomass
' directory of 2MASS Kit,
create symbolic links to downloaded data files.
$ cd twomass $ ln -s (somewhere)/twomass_data/*.db.txt .
Edit the Makefile:
$ vi Makefile
Two tables are created for 2MASS PSC: "Main
Table," and "Dedicated Positional Search Table."
Basically all positional searches use
the Dedicated Positional Search Table via stored functions.
With other searches the vanilla
SELECT ... FROM ... WHERE ...
syntax is used for searching the Main Table.
However, if you don't specify coordinates and force a search with
flux or the like you will not be able to search at a reasonable speed,
even if you did create an index, unless the search space is narrowed
down to several thousand near the center of the distribution using a
"LIMIT
" clause.
In the Makefile,
INDICES
specifies which columns in Main Table are populated with indices.
The default is j_m,h_m,k_m
,
which creates indices for flux j, h, and k. Add column names as needed.
However, do not add objid
because an index is created for it even if unspecified.
And then, run make.
$ make
SQL files will be generated.
As a normal user in UNIX log in to the database "2MASS" as 'postgres' (superuser in PostgreSQL) to register the table spaces:
$ psql -U postgres 2MASS Password for user postgres: xxxxxx psql (8.4.5) Type "help" for help. 2MASS=# \i twomass_create_tablespace.sql CREATE TABLESPACE CREATE TABLESPACE CREATE TABLESPACE CREATE TABLESPACE CREATE TABLESPACE CREATE TABLESPACE 2MASS=# \q
As a normal user in UNIX log in to the database "2MASS" as 'admin' to register tables:
$ psql -U admin 2MASS Password for user admin: xxxxxx psql (8.4.5) Type "help" for help. 2MASS=> \i twomass_create_table.sql CREATE TABLE CREATE TABLE
Note: From here on in we will be using commands that take quite a long time to execute. I will include the time required with an Adaptec RAID 2405 + S-ATA2 HDD x 2 (RAID 1), but twice to ten times longer could be required depending on the environment used.
The next step involving registering data and creating the primary keys in the Main Table. This will take several hours (five hours or more) to complete:
2MASS=> \i twomass_main_copy.sql SET psql:twomass_main_copy.sql:xxxxxxxxx: NOTICE: ALTER TABLE / ADD PRIMARY KEY will create implicit index "twomass_pkey" for table "twomass" ALTER TABLE SET
The following type of message may occasionally seen, which can be safely ignored:
LOG: checkpoints are occurring too frequently (7 seconds apart) HINT: Consider increasing the configuration parameter "checkpoint_segments".
If the data registration process does not finish within three days or so you may have some problematic BIOS settings with the S-ATA interface. Refer to FAQ section for more details.
Create the table, register the data, and create indices as shown below. It will take about 30 minutes to register the data and 80 minutes to create the indices:
2MASS=> \i twomass_eqi_create_table.sql 2MASS=> \i twomass_eqi_copy.sql 2MASS=> \i twomass_eqi_create_index.sql
The floowing type of message may occasionally be seen during the index creation, which can be safely ignored:
ERROR: canceling autovacuum task CONTEXT: automatic analyze of table "xxxxxxxx"
Register stored functions for 2MASS:
2MASS=> \i twomass_create_function.sql
2MASS=> \i twomass_main_grant.sql 2MASS=> \i twomass_eqi_grant.sql
If you are using something other than coordinates as the top priority search conditions of you may wish to create indices in the Main Table. It will take several hours to complete with standard settings.
2MASS=> \i twomass_main_create_index.sql
Indices can be added to the Main Table later if preferred. After adding column names to
INDICES
in the Makefile run, make and then execute "twomass_main_create_index.sql
" as above.
In this case ensure to run
"VACUUM ANALYZE Twomass;
"
after the indices have been created.
At long last we have arrived at the end of our command sequence. Enter this command and wait a few hours, upon which you should then be able to perform searches. Congratulations!
2MASS=> \i twomass_vacuum.sql
During the execution of VACUUM ANALYZE without arguments
you will see a lot of messages like
"WARNING: skipping "pg_type" --- only superuser or database owner can vacuum it
",
but you can safely ignore them.