scispace - formally typeset
Search or ask a question
Proceedings ArticleDOI

Architectural Optimization of Large Scale Astronomical Data

01 Jul 2019-pp 1-6
TL;DR: The techniques used in the development of the proposed system including data migration, prefetching of data and minimizing the time trade-off, address all existing challenges and show a significant improvement in the overall performance of the system.
Abstract: Satellites all around the globe generate huge quantities of data periodically on a daily basis. The management of such large quantities of data poses significant problems ranging from data storage, retrieval and manipulation. The current scenario of managing Indian Space Research Organisation's (ISRO) ASTROSAT-CZTI satellite data involves an elementary and unsophisticated approach which causes a lot of delay in data retrieval and further causes noticeable data portability issues. The evaluation and research on the above mentioned complication paved a way for us to introduce a new architectural system to solve this complex optimization problem. We propose the creation of an independent application server and an underlying Django REST API based web service which would facilitate the efficient management and retrieval of this data. This also extends the scope, making the system more dynamic and helps in load balancing. The application server helps the system to regulate the turbulent flow of huge data quantities. The Django web framework provides feature distribution which helps manage unorganised data cost effectively. Thus, it removes the possible causes of deterioration of response in the existing system. The REST API extends the scope of the system by providing umpteen useful features which makes universal interfacing of our system possible. The techniques used in the development of our proposed system including data migration, prefetching of data and minimizing the time trade-off, address all existing challenges and show a significant improvement in the overall performance of the system.
References
More filters
Journal ArticleDOI
TL;DR: The Flexible Image Transport System (FITS) has been in use in the astronomical community for over two decades as mentioned in this paper and a newly updated version of the standard has recently been approved by the International Astronomical Union FITS Working Group.
Abstract: The Flexible Image Transport System { FITS { has been in use in the astronomical community for over two decades. A newly updated version of the standard has recently been approved by the International Astronomical Union FITS Working Group. This new version of the standard appears here in its entirety. As a preface we briefly describe the process by which the standard evolves and revisions are approved, and note two minor changes to NOST 100{2.0 which were adopted by the IAU FWG.

267 citations


"Architectural Optimization of Large..." refers methods in this paper

  • ...ASTROSAT-CZTI(Cadmium Zinc Telluride Imager) [1] data is acquired from the satellite in the FITS (Flexible Image Transport System) [2] format and stored after transforming it into ASCII and JPEG at IUCAA (Inter-University Centre for Astronomy and Astrophysics) [3] servers....

    [...]

Book
24 Oct 2008
TL;DR: This complete guide starts by introducing Python, Django, and Web development concepts, then dives into the Django framework, providing a deep understanding of its major components (models, views, templates), and how they come together to form complete Web applications.
Abstract: Using the simple, robust, Python-based Django framework, you can build powerful Web solutions with remarkably few lines of code. In Python Web Development with Django, three experienced Django and Python developers cover all the techniques, tools, and concepts you need to make the most of Django 1.0, including all the major features of the new release. The authors teach Django through in-depth explanations, plus provide extensive sample code supported with images and line-by-line explanations. Youll discover how Django leverages Pythons development speed and flexibility to help you solve a wide spectrum of Web development problems and learn Django best practices covered nowhere else. Youll build your first Django application in just minutes and deepen your real-world skills through start-to-finish application projects including Simple Web log (blog) Online photo gallery Simple content management system Ajax-powered live blogger Online source code sharing/syntax highlighting tool How to run your Django applications on the Google App Engine This complete guide starts by introducing Python, Django, and Web development concepts, then dives into the Django framework, providing a deep understanding of its major components (models, views, templates), and how they come together to form complete Web applications. After a discussion of four independent working Django applications, coverage turns to advanced topics, such as caching, extending the template system, syndication, admin customization, and testing. Valuable reference appendices cover using the command-line, installing and configuring Django, development tools, exploring existing Django applications, the Google App Engine, and how to get more involved with the Django community. Introduction 1 Part I: Getting Started Chapter 1: Practical Python for Django 7 Chapter 2: Django for the Impatient: Building a Blog 57 Chapter 3: Starting Out 77 Part II: Django in Depth Chapter 4: Defining and Using Models 89 Chapter 5: URLs, HTTP Mechanisms, and Views 117 Chapter 6: Templates and Form Processing 135 Part III: Django Applications by Example Chapter 7: Photo Gallery 159 Chapter 8: Content Management System 181 Chapter 9: Liveblog 205 Chapter 10: Pastebin 221 Part IV: Advanced Django Techniques and Features Chapter 11: Advanced Django Programming 235 Chapter 12: Advanced Django Deployment 261 Part V: Appendices Appendix A: Command Line Basics 285 Appendix B: Installing and Running Django 295 Appendix C: Tools for Practical Django Development 313 Appendix D: Finding, Evaluating, and Using Django Applications 321 Appendix E: Django on the Google App Engine 325 Appendix F: Getting Involved in the Django Project 337 Index 339 Colophon 375

73 citations


"Architectural Optimization of Large..." refers background in this paper

  • ...It removes possible causes for degradation which are currently present in the existing system [23]....

    [...]

Proceedings ArticleDOI
01 Oct 2016
TL;DR: This paper compares SQL and NoSQL databases for a small scale IoT application of water sprinkler system and investigates whether NoSQL performs better than SQL in different scenarios.
Abstract: Internet of Things (IoT) concept has been around in tech world for few years now. IoT focuses on connection of number of smart devices. In near future, IoT will have applications in various domains and these applications are going to produce tremendous amount of data. With the continuous generation of heterogeneous data, problem arises to store, transfer & manage the data efficiently. Traditional database systems used Structured Query Language (SQL) database which has supported all the user requirements along with simplicity, robustness, flexibility, scalability, performance. But the main limitation they are facing is their static schema which is making RDBMS not suitable for IoT applications. On the other hand, NoSQL databases emerging in market have claimed to perform better than SQL database. The NoSQL databases are non-relational, schema free, no joins, easy replication support, horizontally scalable, etc. Does NoSQL perform better than SQL in all application scenarios? An effort to answer the same has been made in this paper. This paper compares SQL and NoSQL databases for a small scale IoT application of water sprinkler system and investigates whether NoSQL performs better than SQL in different scenarios.

58 citations


"Architectural Optimization of Large..." refers methods in this paper

  • ...Hence, in our system, we have proposed the use of a MySQL database [13]....

    [...]

Proceedings ArticleDOI
25 Jun 2014
TL;DR: The scheme uses JSON format to transfer data, support SSL/TLS protocol requests, custom parameter, sets or gets the request header, which is good compatibility, easy to use, suitable for application on Android platform.
Abstract: The combination of Web Services and mobile devices will promote the development of mobile applications. Volley framework Google 2013 proposed has the advantages of convenient use and network request faster, but it does not support Web Ser vices. Extension of Volley, to support the Web Services, which ca n facilitate the Web Services application development, but also ca n improve the access performance of Web Services. On the basis of analysis and research of the Volley, Ksoap2 and Java Web Ser vices, through the implementation of the HttpStack interface and the expansion of Json Object Request to realize support for Web Services. The scheme uses JSON format to transfer data, support SSL/TLS protocol requests, custom parameter, sets or gets the request header. This scheme is good compatibility, easy to use, suitable for application on Android platform.

15 citations


"Architectural Optimization of Large..." refers background or methods in this paper

  • ...Methods and ways available to create and implement REST services APIs with Django were studied....

    [...]

  • ...The Web Service in our architecture consists of three main parts: Django Framework, REST API, Google Volley....

    [...]

  • ...The load balancing and dynamic data fetching process employed in the Django and RESTful API allows the mobile application to maintain a lightweight architecture....

    [...]

  • ...In this paper, an architecture for the ASTROSATCZTIDQRs employing a web service based on Django web framework and RESTful API; Application Server consisting of a relational database; A client-side user interface comprising of a mobile interface and desktop interface is propositioned that optimizes the storage and fetching mechanism of the DQRs....

    [...]

  • ...The Android application will communicate with ARJUN via an efficient Django REST Framework based API and fetch data accordingly....

    [...]

Proceedings ArticleDOI
01 Nov 2014
TL;DR: An application of academic information that helps students to access information easily by utilizing web service technology is designed and built and has achieved the goal.
Abstract: Academic Information is very important for college students and should be conveyed to all students equally. But as is well known that a lot of students make the staff of college cannot convey information in a personal one by one on each student. In order to support good communication, synchronized, and quick to students. College facilitates through conventional media announcement. Academic Information that is conveyed conventionally to students through flyers and bulletin board are often distributed ineffectively. Besides that, students are also bothered by having to come to campus every day just to see the lattest announcements on the bulletin board. The purpose of this research is to design and build a multiplatform application that can be accessed through android platform and browser with same based code and allow students to access the latest academic information anywhere and anytime as well as reduce the uneven distribution of information. System is built using Representational State Transfer (REST) web service that handle variety of cross platform application. Data is sent in Javascript Object Notation (Json) format, where client apllication only need to parse data and provide them to students. Based on the results, an application has achieved the goal which is an application of academic information that helps students to access information easily by utilizing web service technology.

12 citations


"Architectural Optimization of Large..." refers methods in this paper

  • ...While referring previous research [8], we were introduced to the concept of fail-safe maintenance using Django framework which we have implemented in our designed architecture and explained in further sections....

    [...]