Batch query a lot of MySQLdump filesHow to change data structure in mysql using mysqldump without deleting...
It took me a lot of time to make this, pls like. (YouTube Comments #1)
Closed form for these polynomials?
Program that converts a number to a letter of the alphabet
What to do when being responsible for data protection in your lab, yet advice is ignored?
How can I improve my fireworks photography?
Why did Bush enact a completely different foreign policy to that which he espoused during the 2000 Presidential election campaign?
Dilemma of explaining to interviewer that he is the reason for declining second interview
Everyone is beautiful
Eww, those bytes are gross
What's the most convenient time of year in the USA to end the world?
What formula could mimic the following curve?
I am on the US no-fly list. What can I do in order to be allowed on flights which go through US airspace?
What's a good word to describe a public place that looks like it wouldn't be rough?
Knowing when to use pictures over words
Can a hotel cancel a confirmed reservation?
Can I become debt free or should I file for bankruptcy? How do I manage my debt and finances?
Could flying insects re-enter the Earth's atmosphere from space without burning up?
Why is button three on trumpet almost never used alone?
How can animals be objects of ethics without being subjects as well?
Why are the books in the Game of Thrones citadel library shelved spine inwards?
How is the Incom shipyard still in business?
What is the etymology of the kanji 食?
Why zero tolerance on nudity in space?
Approaches to criticizing short fiction
Batch query a lot of MySQLdump files
How to change data structure in mysql using mysqldump without deleting filesmysqldump dump file sizemysqldump confusionMysqldump without slow queryHowto restore Trigger filesystem copy (*.TRG and *.TRN files) in addition to mysqldump?MYSQLDUMP --skip-lock-tablesmysqldump errorMysqldump, exclude data from tables by queryMariaDB Query slower after mysqldumpMysql: set variable in mysqldump
I have a website for an event. I set up a cron job to back up the database every hour. The backup command is
mysqldump -u"$DB_USER" -p"$DB_PASS" --database "$DB_NAME" >"${OUTPUT_DIR}/${DB_NAME}.$(date +%Y%m%d-%H%M%S).sql"
# crontab: 30 * * * * /path/to/backup_db.sh
I have 200+ of files like this in the backup directory (pulled from server):
$ ls -1 "$OUTPUT_DIR"
ibug.20190301-133002.sql
ibug.20190301-143001.sql
ibug.20190301-153001.sql
ibug.20190301-163001.sql
ibug.20190301-173001.sql
...
The files aren't very big by themselves, averaging less than 2 MB each but summing up to more than 400 MB.
The top few lines of a dump is
-- MySQL dump 10.14 Distrib 5.5.60-MariaDB, for Linux (x86_64)
--
-- Host: localhost Database: ibug
-- ------------------------------------------------------
-- Server version 5.5.60-MariaDB
I want to perform a series of SELECT
queries on each dump (data analysis). The queries range from simple SELECT COUNT(*)
ones to complex ones with multiple JOIN
s and multiple subqueries (all are SELECT
).
While it's possible, I don't want to write a script to import them one by one and run the queries. I wonder if there is another better approach.
My local machine is running on 10.1.38-MariaDB-0ubuntu0.18.04.1 Ubuntu 18.04
(not exactly MySQL, it's Maria).
I have (I believe) enough skills and knowledge in Python and Bash (and regular expressions) for a moderately complex script to batch or automate the job, not sure if this is relevant, though.
mysql mariadb mysqldump query
New contributor
add a comment |
I have a website for an event. I set up a cron job to back up the database every hour. The backup command is
mysqldump -u"$DB_USER" -p"$DB_PASS" --database "$DB_NAME" >"${OUTPUT_DIR}/${DB_NAME}.$(date +%Y%m%d-%H%M%S).sql"
# crontab: 30 * * * * /path/to/backup_db.sh
I have 200+ of files like this in the backup directory (pulled from server):
$ ls -1 "$OUTPUT_DIR"
ibug.20190301-133002.sql
ibug.20190301-143001.sql
ibug.20190301-153001.sql
ibug.20190301-163001.sql
ibug.20190301-173001.sql
...
The files aren't very big by themselves, averaging less than 2 MB each but summing up to more than 400 MB.
The top few lines of a dump is
-- MySQL dump 10.14 Distrib 5.5.60-MariaDB, for Linux (x86_64)
--
-- Host: localhost Database: ibug
-- ------------------------------------------------------
-- Server version 5.5.60-MariaDB
I want to perform a series of SELECT
queries on each dump (data analysis). The queries range from simple SELECT COUNT(*)
ones to complex ones with multiple JOIN
s and multiple subqueries (all are SELECT
).
While it's possible, I don't want to write a script to import them one by one and run the queries. I wonder if there is another better approach.
My local machine is running on 10.1.38-MariaDB-0ubuntu0.18.04.1 Ubuntu 18.04
(not exactly MySQL, it's Maria).
I have (I believe) enough skills and knowledge in Python and Bash (and regular expressions) for a moderately complex script to batch or automate the job, not sure if this is relevant, though.
mysql mariadb mysqldump query
New contributor
add a comment |
I have a website for an event. I set up a cron job to back up the database every hour. The backup command is
mysqldump -u"$DB_USER" -p"$DB_PASS" --database "$DB_NAME" >"${OUTPUT_DIR}/${DB_NAME}.$(date +%Y%m%d-%H%M%S).sql"
# crontab: 30 * * * * /path/to/backup_db.sh
I have 200+ of files like this in the backup directory (pulled from server):
$ ls -1 "$OUTPUT_DIR"
ibug.20190301-133002.sql
ibug.20190301-143001.sql
ibug.20190301-153001.sql
ibug.20190301-163001.sql
ibug.20190301-173001.sql
...
The files aren't very big by themselves, averaging less than 2 MB each but summing up to more than 400 MB.
The top few lines of a dump is
-- MySQL dump 10.14 Distrib 5.5.60-MariaDB, for Linux (x86_64)
--
-- Host: localhost Database: ibug
-- ------------------------------------------------------
-- Server version 5.5.60-MariaDB
I want to perform a series of SELECT
queries on each dump (data analysis). The queries range from simple SELECT COUNT(*)
ones to complex ones with multiple JOIN
s and multiple subqueries (all are SELECT
).
While it's possible, I don't want to write a script to import them one by one and run the queries. I wonder if there is another better approach.
My local machine is running on 10.1.38-MariaDB-0ubuntu0.18.04.1 Ubuntu 18.04
(not exactly MySQL, it's Maria).
I have (I believe) enough skills and knowledge in Python and Bash (and regular expressions) for a moderately complex script to batch or automate the job, not sure if this is relevant, though.
mysql mariadb mysqldump query
New contributor
I have a website for an event. I set up a cron job to back up the database every hour. The backup command is
mysqldump -u"$DB_USER" -p"$DB_PASS" --database "$DB_NAME" >"${OUTPUT_DIR}/${DB_NAME}.$(date +%Y%m%d-%H%M%S).sql"
# crontab: 30 * * * * /path/to/backup_db.sh
I have 200+ of files like this in the backup directory (pulled from server):
$ ls -1 "$OUTPUT_DIR"
ibug.20190301-133002.sql
ibug.20190301-143001.sql
ibug.20190301-153001.sql
ibug.20190301-163001.sql
ibug.20190301-173001.sql
...
The files aren't very big by themselves, averaging less than 2 MB each but summing up to more than 400 MB.
The top few lines of a dump is
-- MySQL dump 10.14 Distrib 5.5.60-MariaDB, for Linux (x86_64)
--
-- Host: localhost Database: ibug
-- ------------------------------------------------------
-- Server version 5.5.60-MariaDB
I want to perform a series of SELECT
queries on each dump (data analysis). The queries range from simple SELECT COUNT(*)
ones to complex ones with multiple JOIN
s and multiple subqueries (all are SELECT
).
While it's possible, I don't want to write a script to import them one by one and run the queries. I wonder if there is another better approach.
My local machine is running on 10.1.38-MariaDB-0ubuntu0.18.04.1 Ubuntu 18.04
(not exactly MySQL, it's Maria).
I have (I believe) enough skills and knowledge in Python and Bash (and regular expressions) for a moderately complex script to batch or automate the job, not sure if this is relevant, though.
mysql mariadb mysqldump query
mysql mariadb mysqldump query
New contributor
New contributor
edited 2 mins ago
SELECT iBug FROM users
New contributor
asked 8 mins ago
SELECT iBug FROM usersSELECT iBug FROM users
1013
1013
New contributor
New contributor
add a comment |
add a comment |
0
active
oldest
votes
Your Answer
StackExchange.ready(function() {
var channelOptions = {
tags: "".split(" "),
id: "182"
};
initTagRenderer("".split(" "), "".split(" "), channelOptions);
StackExchange.using("externalEditor", function() {
// Have to fire editor after snippets, if snippets enabled
if (StackExchange.settings.snippets.snippetsEnabled) {
StackExchange.using("snippets", function() {
createEditor();
});
}
else {
createEditor();
}
});
function createEditor() {
StackExchange.prepareEditor({
heartbeatType: 'answer',
autoActivateHeartbeat: false,
convertImagesToLinks: false,
noModals: true,
showLowRepImageUploadWarning: true,
reputationToPostImages: null,
bindNavPrevention: true,
postfix: "",
imageUploader: {
brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
allowUrls: true
},
onDemand: true,
discardSelector: ".discard-answer"
,immediatelyShowMarkdownHelp:true
});
}
});
SELECT iBug FROM users is a new contributor. Be nice, and check out our Code of Conduct.
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fdba.stackexchange.com%2fquestions%2f231156%2fbatch-query-a-lot-of-mysqldump-files%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
0
active
oldest
votes
0
active
oldest
votes
active
oldest
votes
active
oldest
votes
SELECT iBug FROM users is a new contributor. Be nice, and check out our Code of Conduct.
SELECT iBug FROM users is a new contributor. Be nice, and check out our Code of Conduct.
SELECT iBug FROM users is a new contributor. Be nice, and check out our Code of Conduct.
SELECT iBug FROM users is a new contributor. Be nice, and check out our Code of Conduct.
Thanks for contributing an answer to Database Administrators Stack Exchange!
- Please be sure to answer the question. Provide details and share your research!
But avoid …
- Asking for help, clarification, or responding to other answers.
- Making statements based on opinion; back them up with references or personal experience.
To learn more, see our tips on writing great answers.
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fdba.stackexchange.com%2fquestions%2f231156%2fbatch-query-a-lot-of-mysqldump-files%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown