Ygthkb

Program that converts a number to a letter of the alphabet

How is the Incom shipyard still in business?

How Internet Communication Works on a Coaxial Cable

What is the wife of a henpecked husband called?

Why zero tolerance on nudity in space?

What makes the Forgotten Realms "forgotten"?

Is there a better way to make this?

Tikzing a circled star

What is better: yes / no radio, or simple checkbox?

Strange Sign on Lab Door

Does Windows 10's telemetry include sending *.doc files if Word crashed?

Why can a 352GB NumPy ndarray be used on an 8GB memory macOS computer?

Can polymorphing monsters spam their ability to effectively give themselves a massive health pool?

Is it safe to try charging my laptop with a USB-C PD charger that has less wattage than recommended?

Avoiding morning and evening handshakes

Can we use the stored gravitational potential energy of a building to produce power?

Does the "particle exchange" operator have any validity?

What's the most convenient time of year in the USA to end the world?

How to acknowledge an embarrassing job interview, now that I work directly with the interviewer?

Overfitting and Underfitting

What to do when being responsible for data protection in your lab, yet advice is ignored?

How would an AI self awareness kill switch work?

Eww, those bytes are gross

How would one buy a used TIE Fighter or X-Wing?

Find efficient way of migration 2TB GBK table in SybaseIQ to UTF-8 files

Oracle 10g Storage MigrationTell MySQL to start using utf-8 encoding without `convert to`ing itServer default collation change or not changeHow do I convert unicode data from UTF-16LE to UTF-8 when migrating from SQL Server to MySQL?Handling lossy data before character- set migrationHow to create and populate a PostgreSQL table from CSV automatically?Missing Oracle Schema in SSMA Object Explorermysqldump collation issuesCan't set up Unicode supportProblems during migration from SQL Server to MySQL via MySQL Workbench

1

My project need to migrate more than 2TB raw data from SybaseIQ to another Database. SybaseIQ using GBK, the target Database using UTF-8, the target database can only load from UTF-8 files.

We use The Extract Options to extract data to GBK files first, then convert data into UTF-8, the converting is very slow because it's single processor only.

Can anyone tell me the efficient way of extract data from SybaseIQ and convert charset and encode to UTF-8?

migration sybase utf-8 sybaseiq

share|improve this question

asked Oct 11 '14 at 1:24

Jaugar ChangJaugar Chang

1085

bumped to the homepage by Community♦ 4 mins ago

This question has answers that may be good or bad; the system has marked it active so that they can be reviewed.

add a comment | 

1

My project need to migrate more than 2TB raw data from SybaseIQ to another Database. SybaseIQ using GBK, the target Database using UTF-8, the target database can only load from UTF-8 files.

We use The Extract Options to extract data to GBK files first, then convert data into UTF-8, the converting is very slow because it's single processor only.

Can anyone tell me the efficient way of extract data from SybaseIQ and convert charset and encode to UTF-8?

migration sybase utf-8 sybaseiq

share|improve this question

asked Oct 11 '14 at 1:24

Jaugar ChangJaugar Chang

1085

bumped to the homepage by Community♦ 4 mins ago

This question has answers that may be good or bad; the system has marked it active so that they can be reviewed.

add a comment | 

My project need to migrate more than 2TB raw data from SybaseIQ to another Database. SybaseIQ using GBK, the target Database using UTF-8, the target database can only load from UTF-8 files.

We use The Extract Options to extract data to GBK files first, then convert data into UTF-8, the converting is very slow because it's single processor only.

Can anyone tell me the efficient way of extract data from SybaseIQ and convert charset and encode to UTF-8?

migration sybase utf-8 sybaseiq

share|improve this question

asked Oct 11 '14 at 1:24

Jaugar ChangJaugar Chang

1085

share|improve this question

asked Oct 11 '14 at 1:24

Jaugar ChangJaugar Chang

1085

share|improve this question

asked Oct 11 '14 at 1:24

Jaugar ChangJaugar Chang

1085

asked Oct 11 '14 at 1:24

Jaugar ChangJaugar Chang

1085

asked Oct 11 '14 at 1:24

Jaugar ChangJaugar Chang

1085

1 Answer
1

active

oldest

votes

0

I've promoted about 2X speed by named pipe.

First step:(create a named pipe)

mkfifo p1

Second step:(waiting to convert)

cat p1| iconv -f GBK -t UTF-8 > target.file

Last step:(export to p1, see the sql scripts below)

SET OPTION TEMP_EXTRACT_NAME1 = 'p1';

SELECT * FROM MY_TABLE;

SET OPTION TEMP_EXTRACT_NAME1 = '';

share|improve this answer

answered Oct 11 '14 at 4:44

Jaugar ChangJaugar Chang

1085

add a comment | 

Your Answer

StackExchange.ready(function() {
var channelOptions = {
tags: "".split(" "),
id: "182"
};
initTagRenderer("".split(" "), "".split(" "), channelOptions);

StackExchange.using("externalEditor", function() {
// Have to fire editor after snippets, if snippets enabled
if (StackExchange.settings.snippets.snippetsEnabled) {
StackExchange.using("snippets", function() {
createEditor();
});
}
else {
createEditor();
}
});

function createEditor() {
StackExchange.prepareEditor({
heartbeatType: 'answer',
autoActivateHeartbeat: false,
convertImagesToLinks: false,
noModals: true,
showLowRepImageUploadWarning: true,
reputationToPostImages: null,
bindNavPrevention: true,
postfix: "",
imageUploader: {
brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
allowUrls: true
},
onDemand: true,
discardSelector: ".discard-answer"
,immediatelyShowMarkdownHelp:true
});


}
});

draft saved

draft discarded

Sign up or log in

StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});

Sign up using Google

Sign up using Facebook

Sign up using Email and Password

Post as a guest

Name

Email

Required, but never shown

StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fdba.stackexchange.com%2fquestions%2f78935%2ffind-efficient-way-of-migration-2tb-gbk-table-in-sybaseiq-to-utf-8-files%23new-answer', 'question_page');
}
);

Post as a guest

Name

Email

Required, but never shown

0

I've promoted about 2X speed by named pipe.

First step:(create a named pipe)

mkfifo p1

Second step:(waiting to convert)

cat p1| iconv -f GBK -t UTF-8 > target.file

Last step:(export to p1, see the sql scripts below)

SET OPTION TEMP_EXTRACT_NAME1 = 'p1';

SELECT * FROM MY_TABLE;

SET OPTION TEMP_EXTRACT_NAME1 = '';

share|improve this answer

answered Oct 11 '14 at 4:44

Jaugar ChangJaugar Chang

1085

add a comment | 

0

I've promoted about 2X speed by named pipe.

First step:(create a named pipe)

mkfifo p1

Second step:(waiting to convert)

cat p1| iconv -f GBK -t UTF-8 > target.file

Last step:(export to p1, see the sql scripts below)

SET OPTION TEMP_EXTRACT_NAME1 = 'p1';

SELECT * FROM MY_TABLE;

SET OPTION TEMP_EXTRACT_NAME1 = '';

share|improve this answer

answered Oct 11 '14 at 4:44

Jaugar ChangJaugar Chang

1085

add a comment | 

I've promoted about 2X speed by named pipe.

First step:(create a named pipe)

mkfifo p1

Second step:(waiting to convert)

cat p1| iconv -f GBK -t UTF-8 > target.file

Last step:(export to p1, see the sql scripts below)

SET OPTION TEMP_EXTRACT_NAME1 = 'p1';

SELECT * FROM MY_TABLE;

SET OPTION TEMP_EXTRACT_NAME1 = '';

share|improve this answer

answered Oct 11 '14 at 4:44

Jaugar ChangJaugar Chang

1085

mkfifo p1

cat p1| iconv -f GBK -t UTF-8 > target.file

SET OPTION TEMP_EXTRACT_NAME1 = 'p1';

SELECT * FROM MY_TABLE;

SET OPTION TEMP_EXTRACT_NAME1 = '';

share|improve this answer

answered Oct 11 '14 at 4:44

Jaugar ChangJaugar Chang

1085

answered Oct 11 '14 at 4:44

Jaugar ChangJaugar Chang

1085

answered Oct 11 '14 at 4:44

Jaugar ChangJaugar Chang

1085