Kopernik: Difference between revisions
No edit summary (change visibility) |
|||
| (81 intermediate revisions by the same user not shown) | |||
| Line 1: | Line 1: | ||
== '''''Week One''''' == |
|||
__TOC__ |
|||
== Introduction to Kopernik Observatory== |
|||
Kopernik Observatory & Science Center is a non-profit organization which is opened in 1974. Located atop a 1720 foot hill in Vestal, NY, the Observatory has been one of the best equipped public observatories in the Northeast United States for nearly 40 years. It provides educational program for students and their families. |
|||
== Information about Previous Project == |
|||
=== Relational Database === |
|||
In the previous project, previous team created a new database and combined four databases into one and implemented ten tables inside of the database. As it is mentioned in their final report, it is relation-based SQL file and it is implemented in MySQL environment. The previous project team only worked on the Kopernik Talent Search database which has 4894 rows (including some duplications). We contacted with a team member of the previous project Nagihan Cay and learned that they used Oracle MySQL Workbench Tool to extract the data from excel tables to import them into MySQL. To give general idea about the previous project, reader can find the Entity-Relationship Model (Figure 1[1]) in the appendices part. Since the table names and its attributes might be changed in the future, the document does not consists of their information. |
|||
=== Web Application === |
|||
By reading the previous project’s user manual, we found out that they designed some new General User Interface (GUI) and also they made some improvements on the web pages which are created by the project team before them. This improvements involve more input fields and student search boxes. They created several user-friendly web pages and explained these web pages purposes in their final report. A screenshot of an example for a web page (Figure 2[2]) can be found in the appendices part. |
|||
== Information about Current Project == |
|||
=== Objective === |
|||
Kopernik Observatory gives tens of educational programs to hundreds of students. The information about the students, their families, class, and memberships requires a big database which cannot be handled manually. Their old database is implemented in FileMaker Pro7. Although it is an effective technology for small companies, Kopernik has lots of data to store. The high amount of data makes the FileMaker Pro7 very inefficient. Not the database structure itself, also the user interface of the FileMaker is very complicated. It has lots of textboxes and unordered lists which makes users to confuse about what they are doing, especially the unexperienced users. |
|||
Kopernik Science & Observatory Center Database and Web Application Project is created by previous project team to improve the Kopernik’s current database system and its interface. To achieve this, the previous team started implementing a new relational database structure and a web application. Our project has three stages: |
|||
'''Title:''' Weekly Updates on Kopernik |
|||
#Database Stage – In this stage, we will start examining the database structure which is created by the previous project team, then we will continue from where they left off. Our main priority will be the reconstruct of the current database system. |
|||
#Web Application Page Stage – In this stage, we will check the previous project team’s web pages and decided the adequacy of the current web pages. If there will be a need for more pages, we will create them. |
|||
#Miscellaneous Stage – In the miscellaneous stage we will try to fulfill the extra specifications. This stage can rather be done or left out according to our schedule. If we can finish the first two stages, we will focus on the more “unimportant” functions or modules such as “Spell checking”, “go back buttons” etc. |
|||
'''Date:''' 02/06/2022 |
|||
=== Project Scope === |
|||
'''Subject:''' Getting Started |
|||
'''Attendance:''' Baris Izar, oguz Kircan, Ata Akin, Professor Jones, Andrew Deskur |
|||
'''Meeting Agenda''' |
|||
- Satisfying necessary paperwork before starting the Kopernik Project. |
|||
'''Current Status''' |
|||
- Prof. Jones shared the information about the project with us in our first meeting. |
|||
- Task distribution was made, including who worked in which part of the project. |
|||
- Ata Akin will continue to look at the meteor m2 weather satellite that he has been working on in the previous period. |
|||
- Baris Izar will display the images obtained from the satellites that will work on the FTP server on its website. |
|||
- Oguz Kircan will provide support to both sides and monitor the progress of the project. |
|||
'''Accomplishments''' |
|||
- Set up a time for weekly meetings with Prof. Jones, Andrew Deskur, Ata Akin , Oguz Kircan and Baris Izar. |
|||
- We (Oğuz and Barış) started to study Selenium and Wordpress. |
|||
- Ata informed us about the project more in depth. |
|||
== '''''Week Two''''' == |
|||
'''Title:''' Meeting with Andrew Deskur |
|||
'''Date:''' 02/13/2022 |
|||
'''Subject:''' Kopernik Project Requirements |
|||
'''Attendance:''' Oguz Kircan, Baris Izar, Ata Akin, Professor Jones, Andrew Deskur |
|||
'''Meeting Agenda''' |
|||
- We showed what we did in the past week. |
|||
- Ata was able to execute code for capturing the images. |
|||
- Oguz and Baris found the tutorial about word press and the raspberry pi to better understand and study. |
|||
'''Current Status''' |
|||
- After open the weather folder after you got in when you called predict software it asks for Ground station location I already entered the values when you did that once you do not need to do it again. Which helps to predict software to understand your location and when the satellite passes close to ground station the program will know. |
|||
'''Accomplishments''' |
|||
- |
|||
== '''''Week Three''''' == |
|||
'''Title:''' Meeting with Andrew Deskur |
|||
'''Date:''' 02/20/2022 |
|||
'''Subject:''' Kopernik Project Requirements |
|||
'''Attendance:''' Oguz Kircan, Baris Izar, Ata Akin, Professor Jones, Andrew Deskur |
|||
'''Accomplishments:''' |
|||
- Baris connected to the VNC viewer without a problem. |
|||
- find_images() method inside the ftp_allfiles.py, was not functioning. Now, it's debugged and working. |
|||
- ftp_allfiles.py runs without giving any errors. It creates a ftp object and connects to the host through it. |
|||
- The data transfer is also will be through that object. It is also possible to change the directory |
|||
'''Problems''' |
|||
- Even though the code seems working, we are not sure how to test if ftp_allfiles.py really uploades the images. |
|||
'''To Do''' |
|||
- Test if ftp_allfiles.py successfully handles the file transfer. |
|||
== '''''Week Five''''' == |
|||
'''Accomplishments:''' |
|||
- |
|||
'''Problems:''' |
|||
New NOAA GOES server is still freezing and too laggy to use. We are trying to understand what's causing it. |
|||
Things we had checked so far: |
|||
* Running out of memory |
|||
There's still 11 gb available memory. So, this is not the problem. |
|||
* Operating System that VNC is installed on |
|||
It freezes on both Windows and MAC. So, it is not likely. |
|||
* Missing updates |
|||
There are two packages that need upgrading. They may be the cause. |
|||
* Hit:1 http://archive.raspberrypi.org/debian bullseye InRelease |
|||
* Hit:2 http://raspbian.raspberrypi.org/raspbian bullseye InRelease |
|||
We did not update them yet. |
|||
* CPU Usage |
|||
When we checked the CPU usage of the softwares', we saw |
|||
that their CPU usage mostly differs between 15-40 percent. |
|||
At the beginning, we thought this would be the problem. Yet, |
|||
it was the same with the NOAA APT server and it seems |
|||
working properly. |
|||
* Softwares |
|||
Problem might be caused by a software, which is installed on New GOES and not installed on Kopernik NOAA APT. There are only two softwares meeting the criteria: |
|||
* ImageMagick |
|||
Deleting ImageMagick did not effect the freezing problem. |
|||
* GNU Radio Companion |
|||
It's still trying to remove the packages. |
|||
'''TO DO:''' |
|||
* Find what's causing the abnormal lag on NOAA GOES server and solve it. |
|||
* Try to implement the new Meteor-M 2 satellite imagery capture setup. |
|||
== '''''Week Seven''''' == |
|||
'''Accomplishments:''' |
|||
* Raspberry NOAA V2's installation is complete and it is ready to use. |
|||
* We were getting an error saying that "No package matching "libjpeg9" is available." This took the most of our time. In order to solve that, we comment out the dependencies including libjpeg9; and install libjpeg62-turbo manually instead. |
|||
* We did the same for the package "predict" as well. |
|||
* During the installation, we follow the following link: https://github.com/jekhokie/raspberry-noaa-v2 |
|||
* To overcome the "No package matching ..... is available error", we follow the following link: https://github.com/jekhokie/raspberry-noaa-v2/issues/515 |
|||
'''TO DO:''' |
|||
* We will observe the pi to see is it working or not or is it freezing. |
|||
== '''''Week Eight - 04-04''''' == |
|||
'''Accomplishments:''' |
|||
* We got the antenna from Drew and set it up. |
|||
* We updated the system according to the Drew's email. |
|||
The station information (predict): |
|||
Call Sign: K2ZRO |
|||
Latitude : 42°00.14' If you need decimal degrees use: 42.003889 |
|||
Longitude: 76°02.01' (76.033611 or - 76.033611 |
|||
Altitude : 521 meters |
|||
Settings.yml: |
|||
Latitude : 42.003889 |
|||
Longitude : -76.033611 |
|||
Altitude : 521 |
|||
Timezone offset: -4 |
|||
Ntp server: time-a-g.nist.gov |
|||
meteor_receiver: 'rtl_fm' |
|||
noaa_receiver: 'rtl_fm' |
|||
'''TO DO:''' |
|||
* We will check where the images are saved and see if the new NOAA V2 is working. |
|||
* We will check the after installation part of the jekhokie's github repository and see if we could find any other useful feature to implement. |
|||
== '''''Week Ten- 17-04''''' == |
|||
'''Accomplishments:''' |
|||
* We installed "fbi" to view the images on Raspberry Pi Lite and now, we are able to view them. More info can be found on the following link. |
|||
https://www.raspberrypi-spy.co.uk/2017/02/how-to-display-images-on-raspbian-command-line-with-fbi/ |
|||
* We connected the Raspberry Pi to ssh, using PuTTy, so that we could use multiple terminals to make various tests. |
|||
* We test the reception, which will validate if the antenna, reception line, reception hardware and software is working correctly. We should've heard a voice, but there wasn't any. |
|||
* We checked that if the antenna is connected to the rtl fm radio, using a headphone. It seems okay. |
|||
* We checked the schedule with "crontab - l". All seems fine. |
|||
* If the photo is captured in daylight, our sum_min_elevation variable must be belove the sun angle. In the last capture, the sun angle was 25, and our sum_min_elevation is 6. So, this seems fine as well. |
|||
'''Problems:''' |
|||
* All the satellite images are completely red. |
|||
* We are trying to add the functionality to pushing all captured, processed images to a destination email address. We could not manage to do that yet. |
|||
'''TO DO:''' |
|||
* We will figure out what's causing the images to become red. |
|||
* We will add the functionality to push the captured images as email. |
|||
* We will make further research for the reception. |
|||
*We will check the noise level. The process will take five hours and produce a heatmap waterfall image. |
|||
== '''''Week Eleven - 24-04''''' == |
|||
'''Accomplishments:''' |
|||
* Added a new file to the NOAA APT |
|||
* New python file wpbot2 is doing to the same thing as wpbot.py but the only difference is it loads images to the NOAA15 folder instead of the weather folder, and also deletes the previous files. |
|||
* We setup the new Raspberry Pi Lite at Kopernik Observatory. |
|||
'''TO DO:''' |
|||
* We'll check if the images are still red. |
|||
== '''''Week Twelve - 02-05''''' == |
|||
'''Accomplished''' |
|||
* We managed to display the images on PC, using the web panel, which contains the images from passes, and some information regarding the schedules and the satellites' locations. The meteor images are seem pretty clear. |
|||
* In order to use the web panel, while you are connected the same internet connection with pi and with your PC, you need to connect pi's local ip adress. In this case we connected to " https://192.168.1.240/" which directs us to web panel. You can change the web panel settings from raspberry-noaa-v2/config/settings.yml". |
|||
* We managed to get the images from pi to our PC. This process can be done in two ways. One does not require any software, yet you can only send one image at a time to your PC. More info can be found on the following video: |
|||
https://www.youtube.com/watch?v=rsov0FEhZtk&ab_channel=AbhayKant |
|||
* The other option to transfer files between pi and Pc, is an application called "WinSCP". We used this one to trasnfer all the previous images to our local PC. The software can be downloaded from the following link: |
|||
https://winscp.net/eng/index.php |
|||
* We managed to get the pictures according to the noaa satellite numbers and upload them to the website with their noaa satellite numbers. (Baris' part) |
|||
'''Problems''' |
|||
* Only the meteor images are there. There are no images from any NOAA satellite, only the spectograms. |
|||
'''To Do:''' |
|||
* Understand why there are no NOAA satellite images. |
|||
* We'll figure out how to send the images to the website. |
|||
Latest revision as of 14:57, 4 May 2022
Week One
Title: Weekly Updates on Kopernik
Date: 02/06/2022
Subject: Getting Started
Attendance: Baris Izar, oguz Kircan, Ata Akin, Professor Jones, Andrew Deskur
Meeting Agenda
- Satisfying necessary paperwork before starting the Kopernik Project.
Current Status
- Prof. Jones shared the information about the project with us in our first meeting.
- Task distribution was made, including who worked in which part of the project.
- Ata Akin will continue to look at the meteor m2 weather satellite that he has been working on in the previous period.
- Baris Izar will display the images obtained from the satellites that will work on the FTP server on its website.
- Oguz Kircan will provide support to both sides and monitor the progress of the project.
Accomplishments
- Set up a time for weekly meetings with Prof. Jones, Andrew Deskur, Ata Akin , Oguz Kircan and Baris Izar.
- We (Oğuz and Barış) started to study Selenium and Wordpress.
- Ata informed us about the project more in depth.
Week Two
Title: Meeting with Andrew Deskur
Date: 02/13/2022
Subject: Kopernik Project Requirements
Attendance: Oguz Kircan, Baris Izar, Ata Akin, Professor Jones, Andrew Deskur
Meeting Agenda
- We showed what we did in the past week.
- Ata was able to execute code for capturing the images.
- Oguz and Baris found the tutorial about word press and the raspberry pi to better understand and study.
Current Status
- After open the weather folder after you got in when you called predict software it asks for Ground station location I already entered the values when you did that once you do not need to do it again. Which helps to predict software to understand your location and when the satellite passes close to ground station the program will know.
Accomplishments
-
Week Three
Title: Meeting with Andrew Deskur
Date: 02/20/2022
Subject: Kopernik Project Requirements
Attendance: Oguz Kircan, Baris Izar, Ata Akin, Professor Jones, Andrew Deskur
Accomplishments:
- Baris connected to the VNC viewer without a problem.
- find_images() method inside the ftp_allfiles.py, was not functioning. Now, it's debugged and working.
- ftp_allfiles.py runs without giving any errors. It creates a ftp object and connects to the host through it.
- The data transfer is also will be through that object. It is also possible to change the directory
Problems
- Even though the code seems working, we are not sure how to test if ftp_allfiles.py really uploades the images.
To Do
- Test if ftp_allfiles.py successfully handles the file transfer.
Week Five
Accomplishments:
-
Problems:
New NOAA GOES server is still freezing and too laggy to use. We are trying to understand what's causing it.
Things we had checked so far:
- Running out of memory
There's still 11 gb available memory. So, this is not the problem.
- Operating System that VNC is installed on
It freezes on both Windows and MAC. So, it is not likely.
- Missing updates
There are two packages that need upgrading. They may be the cause.
* Hit:1 http://archive.raspberrypi.org/debian bullseye InRelease * Hit:2 http://raspbian.raspberrypi.org/raspbian bullseye InRelease
We did not update them yet.
- CPU Usage
When we checked the CPU usage of the softwares', we saw that their CPU usage mostly differs between 15-40 percent. At the beginning, we thought this would be the problem. Yet, it was the same with the NOAA APT server and it seems working properly.
- Softwares
Problem might be caused by a software, which is installed on New GOES and not installed on Kopernik NOAA APT. There are only two softwares meeting the criteria:
* ImageMagick
Deleting ImageMagick did not effect the freezing problem.
* GNU Radio Companion
It's still trying to remove the packages.
TO DO:
- Find what's causing the abnormal lag on NOAA GOES server and solve it.
- Try to implement the new Meteor-M 2 satellite imagery capture setup.
Week Seven
Accomplishments:
- Raspberry NOAA V2's installation is complete and it is ready to use.
- We were getting an error saying that "No package matching "libjpeg9" is available." This took the most of our time. In order to solve that, we comment out the dependencies including libjpeg9; and install libjpeg62-turbo manually instead.
- We did the same for the package "predict" as well.
- During the installation, we follow the following link: https://github.com/jekhokie/raspberry-noaa-v2
- To overcome the "No package matching ..... is available error", we follow the following link: https://github.com/jekhokie/raspberry-noaa-v2/issues/515
TO DO:
- We will observe the pi to see is it working or not or is it freezing.
Week Eight - 04-04
Accomplishments:
- We got the antenna from Drew and set it up.
- We updated the system according to the Drew's email.
The station information (predict):
Call Sign: K2ZRO
Latitude : 42°00.14' If you need decimal degrees use: 42.003889
Longitude: 76°02.01' (76.033611 or - 76.033611
Altitude : 521 meters
Settings.yml:
Latitude : 42.003889
Longitude : -76.033611
Altitude : 521
Timezone offset: -4
Ntp server: time-a-g.nist.gov
meteor_receiver: 'rtl_fm'
noaa_receiver: 'rtl_fm'
TO DO:
- We will check where the images are saved and see if the new NOAA V2 is working.
- We will check the after installation part of the jekhokie's github repository and see if we could find any other useful feature to implement.
Week Ten- 17-04
Accomplishments:
- We installed "fbi" to view the images on Raspberry Pi Lite and now, we are able to view them. More info can be found on the following link.
https://www.raspberrypi-spy.co.uk/2017/02/how-to-display-images-on-raspbian-command-line-with-fbi/
- We connected the Raspberry Pi to ssh, using PuTTy, so that we could use multiple terminals to make various tests.
- We test the reception, which will validate if the antenna, reception line, reception hardware and software is working correctly. We should've heard a voice, but there wasn't any.
- We checked that if the antenna is connected to the rtl fm radio, using a headphone. It seems okay.
- We checked the schedule with "crontab - l". All seems fine.
- If the photo is captured in daylight, our sum_min_elevation variable must be belove the sun angle. In the last capture, the sun angle was 25, and our sum_min_elevation is 6. So, this seems fine as well.
Problems:
- All the satellite images are completely red.
- We are trying to add the functionality to pushing all captured, processed images to a destination email address. We could not manage to do that yet.
TO DO:
- We will figure out what's causing the images to become red.
- We will add the functionality to push the captured images as email.
- We will make further research for the reception.
- We will check the noise level. The process will take five hours and produce a heatmap waterfall image.
Week Eleven - 24-04
Accomplishments:
- Added a new file to the NOAA APT
- New python file wpbot2 is doing to the same thing as wpbot.py but the only difference is it loads images to the NOAA15 folder instead of the weather folder, and also deletes the previous files.
- We setup the new Raspberry Pi Lite at Kopernik Observatory.
TO DO:
- We'll check if the images are still red.
Week Twelve - 02-05
Accomplished
- We managed to display the images on PC, using the web panel, which contains the images from passes, and some information regarding the schedules and the satellites' locations. The meteor images are seem pretty clear.
- In order to use the web panel, while you are connected the same internet connection with pi and with your PC, you need to connect pi's local ip adress. In this case we connected to " https://192.168.1.240/" which directs us to web panel. You can change the web panel settings from raspberry-noaa-v2/config/settings.yml".
- We managed to get the images from pi to our PC. This process can be done in two ways. One does not require any software, yet you can only send one image at a time to your PC. More info can be found on the following video:
https://www.youtube.com/watch?v=rsov0FEhZtk&ab_channel=AbhayKant
- The other option to transfer files between pi and Pc, is an application called "WinSCP". We used this one to trasnfer all the previous images to our local PC. The software can be downloaded from the following link:
https://winscp.net/eng/index.php
- We managed to get the pictures according to the noaa satellite numbers and upload them to the website with their noaa satellite numbers. (Baris' part)
Problems
- Only the meteor images are there. There are no images from any NOAA satellite, only the spectograms.
To Do:
- Understand why there are no NOAA satellite images.
- We'll figure out how to send the images to the website.