Batch query a lot of MySQLdump filesHow to change data structure in mysql using mysqldump without deleting...

It took me a lot of time to make this, pls like. (YouTube Comments #1)

Closed form for these polynomials?

Program that converts a number to a letter of the alphabet

What to do when being responsible for data protection in your lab, yet advice is ignored?

How can I improve my fireworks photography?

Why did Bush enact a completely different foreign policy to that which he espoused during the 2000 Presidential election campaign?

Dilemma of explaining to interviewer that he is the reason for declining second interview

Everyone is beautiful

Eww, those bytes are gross

What's the most convenient time of year in the USA to end the world?

What formula could mimic the following curve?

I am on the US no-fly list. What can I do in order to be allowed on flights which go through US airspace?

What's a good word to describe a public place that looks like it wouldn't be rough?

Knowing when to use pictures over words

Can a hotel cancel a confirmed reservation?

Can I become debt free or should I file for bankruptcy? How do I manage my debt and finances?

Could flying insects re-enter the Earth's atmosphere from space without burning up?

Why is button three on trumpet almost never used alone?

How can animals be objects of ethics without being subjects as well?

Why are the books in the Game of Thrones citadel library shelved spine inwards?

How is the Incom shipyard still in business?

What is the etymology of the kanji 食?

Why zero tolerance on nudity in space?

Approaches to criticizing short fiction



Batch query a lot of MySQLdump files


How to change data structure in mysql using mysqldump without deleting filesmysqldump dump file sizemysqldump confusionMysqldump without slow queryHowto restore Trigger filesystem copy (*.TRG and *.TRN files) in addition to mysqldump?MYSQLDUMP --skip-lock-tablesmysqldump errorMysqldump, exclude data from tables by queryMariaDB Query slower after mysqldumpMysql: set variable in mysqldump













0















I have a website for an event. I set up a cron job to back up the database every hour. The backup command is



mysqldump -u"$DB_USER" -p"$DB_PASS" --database "$DB_NAME" >"${OUTPUT_DIR}/${DB_NAME}.$(date +%Y%m%d-%H%M%S).sql"

# crontab: 30 * * * * /path/to/backup_db.sh


I have 200+ of files like this in the backup directory (pulled from server):



$ ls -1 "$OUTPUT_DIR"
ibug.20190301-133002.sql
ibug.20190301-143001.sql
ibug.20190301-153001.sql
ibug.20190301-163001.sql
ibug.20190301-173001.sql
...


The files aren't very big by themselves, averaging less than 2 MB each but summing up to more than 400 MB.



The top few lines of a dump is



-- MySQL dump 10.14  Distrib 5.5.60-MariaDB, for Linux (x86_64)
--
-- Host: localhost Database: ibug
-- ------------------------------------------------------
-- Server version 5.5.60-MariaDB


I want to perform a series of SELECT queries on each dump (data analysis). The queries range from simple SELECT COUNT(*) ones to complex ones with multiple JOINs and multiple subqueries (all are SELECT).



While it's possible, I don't want to write a script to import them one by one and run the queries. I wonder if there is another better approach.





My local machine is running on 10.1.38-MariaDB-0ubuntu0.18.04.1 Ubuntu 18.04 (not exactly MySQL, it's Maria).



I have (I believe) enough skills and knowledge in Python and Bash (and regular expressions) for a moderately complex script to batch or automate the job, not sure if this is relevant, though.









share









New contributor




SELECT iBug FROM users is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.

























    0















    I have a website for an event. I set up a cron job to back up the database every hour. The backup command is



    mysqldump -u"$DB_USER" -p"$DB_PASS" --database "$DB_NAME" >"${OUTPUT_DIR}/${DB_NAME}.$(date +%Y%m%d-%H%M%S).sql"

    # crontab: 30 * * * * /path/to/backup_db.sh


    I have 200+ of files like this in the backup directory (pulled from server):



    $ ls -1 "$OUTPUT_DIR"
    ibug.20190301-133002.sql
    ibug.20190301-143001.sql
    ibug.20190301-153001.sql
    ibug.20190301-163001.sql
    ibug.20190301-173001.sql
    ...


    The files aren't very big by themselves, averaging less than 2 MB each but summing up to more than 400 MB.



    The top few lines of a dump is



    -- MySQL dump 10.14  Distrib 5.5.60-MariaDB, for Linux (x86_64)
    --
    -- Host: localhost Database: ibug
    -- ------------------------------------------------------
    -- Server version 5.5.60-MariaDB


    I want to perform a series of SELECT queries on each dump (data analysis). The queries range from simple SELECT COUNT(*) ones to complex ones with multiple JOINs and multiple subqueries (all are SELECT).



    While it's possible, I don't want to write a script to import them one by one and run the queries. I wonder if there is another better approach.





    My local machine is running on 10.1.38-MariaDB-0ubuntu0.18.04.1 Ubuntu 18.04 (not exactly MySQL, it's Maria).



    I have (I believe) enough skills and knowledge in Python and Bash (and regular expressions) for a moderately complex script to batch or automate the job, not sure if this is relevant, though.









    share









    New contributor




    SELECT iBug FROM users is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
    Check out our Code of Conduct.























      0












      0








      0








      I have a website for an event. I set up a cron job to back up the database every hour. The backup command is



      mysqldump -u"$DB_USER" -p"$DB_PASS" --database "$DB_NAME" >"${OUTPUT_DIR}/${DB_NAME}.$(date +%Y%m%d-%H%M%S).sql"

      # crontab: 30 * * * * /path/to/backup_db.sh


      I have 200+ of files like this in the backup directory (pulled from server):



      $ ls -1 "$OUTPUT_DIR"
      ibug.20190301-133002.sql
      ibug.20190301-143001.sql
      ibug.20190301-153001.sql
      ibug.20190301-163001.sql
      ibug.20190301-173001.sql
      ...


      The files aren't very big by themselves, averaging less than 2 MB each but summing up to more than 400 MB.



      The top few lines of a dump is



      -- MySQL dump 10.14  Distrib 5.5.60-MariaDB, for Linux (x86_64)
      --
      -- Host: localhost Database: ibug
      -- ------------------------------------------------------
      -- Server version 5.5.60-MariaDB


      I want to perform a series of SELECT queries on each dump (data analysis). The queries range from simple SELECT COUNT(*) ones to complex ones with multiple JOINs and multiple subqueries (all are SELECT).



      While it's possible, I don't want to write a script to import them one by one and run the queries. I wonder if there is another better approach.





      My local machine is running on 10.1.38-MariaDB-0ubuntu0.18.04.1 Ubuntu 18.04 (not exactly MySQL, it's Maria).



      I have (I believe) enough skills and knowledge in Python and Bash (and regular expressions) for a moderately complex script to batch or automate the job, not sure if this is relevant, though.









      share









      New contributor




      SELECT iBug FROM users is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
      Check out our Code of Conduct.












      I have a website for an event. I set up a cron job to back up the database every hour. The backup command is



      mysqldump -u"$DB_USER" -p"$DB_PASS" --database "$DB_NAME" >"${OUTPUT_DIR}/${DB_NAME}.$(date +%Y%m%d-%H%M%S).sql"

      # crontab: 30 * * * * /path/to/backup_db.sh


      I have 200+ of files like this in the backup directory (pulled from server):



      $ ls -1 "$OUTPUT_DIR"
      ibug.20190301-133002.sql
      ibug.20190301-143001.sql
      ibug.20190301-153001.sql
      ibug.20190301-163001.sql
      ibug.20190301-173001.sql
      ...


      The files aren't very big by themselves, averaging less than 2 MB each but summing up to more than 400 MB.



      The top few lines of a dump is



      -- MySQL dump 10.14  Distrib 5.5.60-MariaDB, for Linux (x86_64)
      --
      -- Host: localhost Database: ibug
      -- ------------------------------------------------------
      -- Server version 5.5.60-MariaDB


      I want to perform a series of SELECT queries on each dump (data analysis). The queries range from simple SELECT COUNT(*) ones to complex ones with multiple JOINs and multiple subqueries (all are SELECT).



      While it's possible, I don't want to write a script to import them one by one and run the queries. I wonder if there is another better approach.





      My local machine is running on 10.1.38-MariaDB-0ubuntu0.18.04.1 Ubuntu 18.04 (not exactly MySQL, it's Maria).



      I have (I believe) enough skills and knowledge in Python and Bash (and regular expressions) for a moderately complex script to batch or automate the job, not sure if this is relevant, though.







      mysql mariadb mysqldump query





      share









      New contributor




      SELECT iBug FROM users is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
      Check out our Code of Conduct.










      share









      New contributor




      SELECT iBug FROM users is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
      Check out our Code of Conduct.








      share



      share








      edited 2 mins ago







      SELECT iBug FROM users













      New contributor




      SELECT iBug FROM users is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
      Check out our Code of Conduct.









      asked 8 mins ago









      SELECT iBug FROM usersSELECT iBug FROM users

      1013




      1013




      New contributor




      SELECT iBug FROM users is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
      Check out our Code of Conduct.





      New contributor





      SELECT iBug FROM users is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
      Check out our Code of Conduct.






      SELECT iBug FROM users is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
      Check out our Code of Conduct.






















          0






          active

          oldest

          votes











          Your Answer








          StackExchange.ready(function() {
          var channelOptions = {
          tags: "".split(" "),
          id: "182"
          };
          initTagRenderer("".split(" "), "".split(" "), channelOptions);

          StackExchange.using("externalEditor", function() {
          // Have to fire editor after snippets, if snippets enabled
          if (StackExchange.settings.snippets.snippetsEnabled) {
          StackExchange.using("snippets", function() {
          createEditor();
          });
          }
          else {
          createEditor();
          }
          });

          function createEditor() {
          StackExchange.prepareEditor({
          heartbeatType: 'answer',
          autoActivateHeartbeat: false,
          convertImagesToLinks: false,
          noModals: true,
          showLowRepImageUploadWarning: true,
          reputationToPostImages: null,
          bindNavPrevention: true,
          postfix: "",
          imageUploader: {
          brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
          contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
          allowUrls: true
          },
          onDemand: true,
          discardSelector: ".discard-answer"
          ,immediatelyShowMarkdownHelp:true
          });


          }
          });






          SELECT iBug FROM users is a new contributor. Be nice, and check out our Code of Conduct.










          draft saved

          draft discarded


















          StackExchange.ready(
          function () {
          StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fdba.stackexchange.com%2fquestions%2f231156%2fbatch-query-a-lot-of-mysqldump-files%23new-answer', 'question_page');
          }
          );

          Post as a guest















          Required, but never shown

























          0






          active

          oldest

          votes








          0






          active

          oldest

          votes









          active

          oldest

          votes






          active

          oldest

          votes








          SELECT iBug FROM users is a new contributor. Be nice, and check out our Code of Conduct.










          draft saved

          draft discarded


















          SELECT iBug FROM users is a new contributor. Be nice, and check out our Code of Conduct.













          SELECT iBug FROM users is a new contributor. Be nice, and check out our Code of Conduct.












          SELECT iBug FROM users is a new contributor. Be nice, and check out our Code of Conduct.
















          Thanks for contributing an answer to Database Administrators Stack Exchange!


          • Please be sure to answer the question. Provide details and share your research!

          But avoid



          • Asking for help, clarification, or responding to other answers.

          • Making statements based on opinion; back them up with references or personal experience.


          To learn more, see our tips on writing great answers.




          draft saved


          draft discarded














          StackExchange.ready(
          function () {
          StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fdba.stackexchange.com%2fquestions%2f231156%2fbatch-query-a-lot-of-mysqldump-files%23new-answer', 'question_page');
          }
          );

          Post as a guest















          Required, but never shown





















































          Required, but never shown














          Required, but never shown












          Required, but never shown







          Required, but never shown

































          Required, but never shown














          Required, but never shown












          Required, but never shown







          Required, but never shown







          Popular posts from this blog

          Szabolcs (Ungheria) Altri progetti | Menu di navigazione48°10′14.56″N 21°29′33.14″E /...

          Discografia di Klaus Schulze Indice Album in studio | Album dal vivo | Singoli | Antologie | Colonne...

          How to make inet_server_addr() return localhost in spite of ::1/128RETURN NEXT in Postgres FunctionConnect to...