Processing of Large Excel Files, Appending data, and extracting specific sheets

Now its made easy to process any large excel files and extract your required sheets by giving the name in input file.

-Use Case: In Telecom networks we have a large set of Export files which will be very difficult to process and append multiple sheets. Now in just one click its made easy.

-Base Tool: Python

-Application: It is application to any large excel files dataset, do not confuse with name.

Feel free to contact me in case of any confusion.

Download: Export & append Selected Sheet from large excel file.rar - Google Drive

LinkedIn: :point_down:

Better to handle it in SQL Server with a stored procedure.

Python provides a wide range of libraries and tools specifically designed for data manipulation and processing, including libraries like pandas for handling Excel files efficiently.

It offers flexibility in terms of data transformation, filtering, and analysis, which may not be as straightforward or flexible when using SQL Server alone.

Python, when used alongside optimized libraries like pandas, can handle large datasets efficiently.

These libraries are designed to process data in-memory and provide vectorized operations, resulting in faster execution times compared to traditional SQL operations.

Although SQL Server and stored procedures have their strengths in handling structured data and complex querying, combining Python’s data processing capabilities with SQL Server’s storage and retrieval features can provide a more comprehensive and flexible solution for handling large Excel files.

Good!

Have you used SQL Server Machine Learning services?

No, I haven’t personally used SQL Server Machine Learning Services.

However, I am aware that SQL Server Machine Learning Services integrates machine learning capabilities directly into SQL Server, allowing you to run R and Python scripts for advanced analytics within the database engine. It enables tasks such as training models, making predictions, and performing statistical analysis using familiar programming languages.

While I don’t have firsthand experience with SQL Server Machine Learning Services, I do understand its purpose and potential benefits.

We have analyzed terabytes of data using SSIS + SQL Server + PBI.

Yes this tool will help majority of people to analyze gigabyte and megabyte complexed data.

Of course for larger datasets we need more optimized solution.

This tool will help telecom engineers specially who are dealing with large set of technology export files or dump files.