PySpark program to generate large dataset with 100 000 columns and 50 million rows

$10-30 USD

Σε Εξέλιξη

Αναρτήθηκε

πάνω από 1 χρόνο πριν

$10-30 USD

Πληρωμή κατά την παράδοση

I need a PySpark program to generate a large dataset with 100 000 columns and 50 million rows. I should be able to set the number of dimension columns (i.e. columns with non-numeric values such as Country, State, Suburb, Product etc). The rest of the columns must be all numerically populated with random floating-point numbers. The program's output needs to save the data to a single parquet or CSV file. I need to be able to set up the dimension values with CSV tables in the format below. Dimension name: Country File name: [login to view URL] File contents: 1,United States 2,United Arab Emerates 3,Saudi Arabia Random numbers must be picked from the file above to populate the dimensions.

PySpark

Spark

Apache Spark

Ταυτότητα εργασίας: 34964959

Σχετικά με την εργασία

3 προτάσεις

Απομακρυσμένη Εργασία

Ενεργός/ή 2 χρόνια πριν

Ψάχνεις τρόπο για να κερδίσεις μερικά χρήματα;

Διεύθυνση Email

Πλεονεκτήματα πλειοδοσίας στο Freelancer

Καθόρισε τον προϋπολογισμό σου και το χρονοδιάγραμμα

Πληρώσου για τη δουλειά σου

Περίγραψε την πρόταση σου

Η εγγραφή και η πλειοδοσία σε εργασίες είναι δωρεάν

Βραβεύτηκε στον/στην:

@bizzarriandrea81

Hi, I'm a senior data scientist and Python developer with more than 8 years experience. As a data scientist, I collaborated with several machine learning engineers and data engineers to design, develop, and evaluate some ML models on large datasets. I have hands-on experience on big data analysis and data visualization. I also have strong cooperation and communication skills. PERSONAL SKILLS 1. Data Science with Python and R 2. Machine Learning and Statistical Analysis with Python and R 3. Data Visualization through Tableau and Grafana 4. Data Analysis with Python(Numpy, Pandas) 5. DBMS(MySQL, Postgresql, MongoDB, Hive, Cassandra) 6. Deep Learning & Computer Vision(OpenCV, Keras, Tensorflow, Pytorch) 7. Natural Language Processing 8. Big data analysis with PySpark "Satisfy the client with my ability and passion" This is my slogan here. I hope you will be interested in me. Thanks.

$30 USD σε 2 ημέρες

5,0

(1 αξιολόγηση)

1,8

3 freelancers δίνουν μια μέση προσφορά $27 USD για αυτή τη δουλειά

@Valuesolutions

Hello, I hope this finds you well. I have just seen your project requiring; Spark PySpark Apache Spark I believe that my 10-year experience in this field is what you need right away. Avoid the headache of looking further. Let's save time and focus on the real task. My proposed timelines and budget are just placeholder and an open for negotiation to increase or decrease as per the full requirements at hand. Allow me to prove how better my review can be. Smile all the way! Click the message button, so initiate the conversation. Regards, Fridah

$20 USD σε 7 ημέρες

0,0

(0 αξιολογήσεις)

0,0

@albanusklo

Dear Client, Greetings and hoping you are doing well, i welcome you to my profile where quality and client satisfaction is the Priority. CERTIFIED EXPERT IN LISTED PROJECT SKILLS & REQUIREMENTS, After KEENLY reading your description and being in possession of all CLEARLY STATED REQUIRED SKILLS (Apache Spark, Spark and PySpark) as this is my area of PROFESSIONAL SPECIALISATION having the above QUALIFICATIONS, i hereby humbly request you to consider my bid for QUALITY and PROFESSIONAL work. ###// No timeline & Quality Excuses// Unlimited Revisions// 100% SATISFACTION GUARANTEED### Message Me We Discuss More About The Project. incase the project is deleted please hire me directly using the following link; https://www.freelancer.com/u/topgradeclubltd THANK-YOU & WELCOME.

$30 USD σε 1 ημέρα