AWS S3 text files to Relational Database

Σε Εξέλιξη

A little background - I am using AWS to manage a mountain bike timing system. Competitors tag on and off a number of different stages and the tag off time minus the tag on time is the stage racetime and the lowest cumulative time over all stages wins.

Out on the racecourse, I am using Raspberry Pi's with RFID readers which collect and store data in text files which automatically upload to the AWS S3 bucket. I currently have two files in there: Stage 1 [url removed, login to view] and Stage 1 Finish.txt.

I need to get these files into a database to join with a competitor registration database so I can ultimately compute racetimes in AWS, extract some analytics and allow competitors to search and compare times using their surname or racing number to find their race times.

In short, I need help converting these text files into a relational database and this process is automated so that when new items are appended to the text files that they update the database.

I think a Data Pipeline is the way to go but happy to listen to alternative recommendations to achieve my ultimate goals as described above.

The text file item/row is semi-colon delimited as follows:

[url removed, login to view];1s;2016/12/30;13:56:[url removed, login to view];00235689

Meaning log information (not needed), 1s (Stage 1 Start), Date, Time to milliseconds and finally the RFID code.

Any help with the Data Pipeline (or another alternative) is required as soon as possible.

Thanks

Steve

Δεξιότητες: Υπηρεσίες Ιστού Amazon, Διαχείριση Βάσεων Δεδομένων, MySQL

Δείτε περισσότερα: translate audio files text, auduio files text, exporting ning database files, relational database video rent, psd examples files text, relational database project, transcribe sound files text, project database files mega retail store, type picture files text money, perl search files text, php mysql relational database frontend, transcribe audio files text, mysql database hold text files, create html files text files, audio files text

ID Εργασίας: #12676953

Ανατέθηκε στον/στην:

jksnetwork

I am highly skilled in the AWS environment. I have been working with AWS for 5 years now and run multiple production workloads across many AWS services. I have both the AWS Devops Engineer and AWS Certified Solutions Περισσότερα

$250 AUD σε 3 μέρες
(15 Αξιολογήσεις)
4.4

20 freelancers κάνουν προσφορές με μέσο όρο $170 γι' αυτή τη δουλειά

poornachand

I have 9 years experience in JAVA, J2EE, WebServices and enough experience in AWS webservices. I am not sure if we can dine the pipeline such that it extracts te file and fills the required columns of a relational dat Περισσότερα

$166 AUD σε 5 μέρες
(25 Αξιολογήσεις)
5.5
ATLASFSD

A proposal has not yet been provided

$150 AUD σε 2 μέρες
(22 Αξιολογήσεις)
5.0
talkwaqar

Please come online for detail. My Skype Id is waqar.tplex I am a highly skilled senior developer with extensive development experience in system design, .NET technology stack including C#, ASP.NET, MVC, Web Api, SQL Περισσότερα

$155 AUD σε 3 μέρες
(15 Αξιολογήσεις)
4.5
whites11

Hello there. I think lambda could be the solution for you. I can take care of this task if you wish I am an experienced developer with strong aws knowledge

$155 AUD σε 3 μέρες
(1 Αξιολόγηση)
3.7
bhawin91

I can write a php script which can load data from text file to database. I would love ot work on something new and good thing ..

$155 AUD σε 3 μέρες
(12 Αξιολογήσεις)
3.8
ChrisLlV

I'm active in graphic design, web and app development for the last 8+ years. I believe in delivering the best possible solution that can be implemented in your business strategies. I'm serving more than 300 clients aro Περισσότερα

$250 AUD σε 7 μέρες
(6 Αξιολογήσεις)
3.8
titukhan13

sir,this is easy task for me.i will complete your work as soon as possible with out any problem.please give me the chance to work with you.waiting for your reply. Thanks

$35 AUD σε 1 μέρα
(4 Αξιολογήσεις)
3.7
$144 AUD σε 3 μέρες
(2 Αξιολογήσεις)
3.4
DevvTime

Hello there i have read your description, i will recommend a web application for this, can you please contact to discuss in details? i am waiting for your pm thanks

$250 AUD σε 3 μέρες
(7 Αξιολογήσεις)
3.5
enterpriseit1

Hi, I have over 7 years of experience as a database developer. I have completed three recent projects. All clients were 100% satisfied as you can see from their reviews. One of the projects I completed recently was Περισσότερα

$170 AUD σε 3 μέρες
(3 Αξιολογήσεις)
2.9
$111 AUD σε 3 μέρες
(8 Αξιολογήσεις)
3.5
explore1

i have similar experience working with S3 and mysql db.

$222 AUD σε 4 μέρες
(2 Αξιολογήσεις)
2.6
fariaakterdia

Hi, I have read through your project description. I will work as per your direction with 100% accuracy. And I will deliver your project before your timeline. I am waiting for your response to discuss more about the pro Περισσότερα

$150 AUD σε 1 μέρα
(1 Αξιολόγηση)
0.6
$222 AUD σε 5 μέρες
(0 Αξιολογήσεις)
0.0
IonRod

Hello! I can do the job u need. I think Python is the best solution for you as it's covering all your needs. On the other hand maybe it makes sense to use logstash to transform and send data to your database and do Περισσότερα

$222 AUD σε 10 μέρες
(0 Αξιολογήσεις)
0.0
$155 AUD σε 3 μέρες
(0 Αξιολογήσεις)
0.0
ebinthomas

Have 4.5 years of experience in Java, REST, Data Structures and Algorithms, SQL and NoSQL(MongoDB, Redis), Spring, Hibernate hence I will be a good help for this project. I had recently worked with Rajhees, C.E.O, MD , Περισσότερα

$155 AUD σε 3 μέρες
(0 Αξιολογήσεις)
0.0
mak94pk

Hi, Firstly, Thank you for reviewing my proposal. I have gone through your requirements and came to know that you need an expert web developer to build your website. I have strong knowledge and 3+ years experience i Περισσότερα

$222 AUD σε 3 μέρες
(0 Αξιολογήσεις)
0.0
ahmad5857

Hi sir, We assure you the quality of service through various checks that are employed to avoid any compromise on quality of work. Let me introduce my team to you and generally what services we offer to our client Περισσότερα

$150 AUD σε 8 μέρες
(0 Αξιολογήσεις)
0.0
jupitor

you don't have to load your data to different database unless you have compelling reasons. Now amazon athena supports query the s3 data like sql queries. Much less expensive and less maintenance than using different da Περισσότερα

$155 AUD σε 3 μέρες
(0 Αξιολογήσεις)
0.0