public class StructTreeWriter extends TreeWriterBase
TreeWriter.FactorybloomFilter, bloomFilterEntry, bloomFilterUtf8, context, createBloomFilter, encryption, fileStatistics, id, indexStatistics, isPresent, rowIndexPosition, schema, stripeColStatistics| Constructor and Description |
|---|
StructTreeWriter(TypeDescription schema,
WriterEncryptionVariant encryption,
WriterContext context) |
| Modifier and Type | Method and Description |
|---|---|
void |
addStripeStatistics(StripeStatistics[] stats)
During a stripe append, we need to handle the stripe statistics.
|
void |
createRowIndexEntry()
Create a row index entry with the previous location and the current
index statistics.
|
long |
estimateMemory()
Estimate how much memory the writer is consuming excluding the streams.
|
void |
flushStreams()
Flush the TreeWriter stream
|
void |
getCurrentStatistics(ColumnStatistics[] output)
Get the current file statistics for each column.
|
long |
getRawDataSize()
Estimate the memory used if the file was read into Hive's Writable
types.
|
void |
prepareStripe(int stripeId)
Set up for the next stripe.
|
void |
writeBatch(ColumnVector vector,
int offset,
int length)
Write the values from the given vector from offset for length elements.
|
void |
writeFileStatistics()
Write the FileStatistics for each column in each encryption variant.
|
void |
writeRootBatch(VectorizedRowBatch batch,
int offset,
int length)
Handle the top level object write.
|
void |
writeStripe(int requiredIndexEntries)
Write the stripe out to the file.
|
getRowIndex, getRowIndexEntry, getStripeStatisticspublic StructTreeWriter(TypeDescription schema, WriterEncryptionVariant encryption, WriterContext context) throws IOException
IOExceptionpublic void writeRootBatch(VectorizedRowBatch batch, int offset, int length) throws IOException
TreeWriterBasewriteRootBatch in interface TreeWriterwriteRootBatch in class TreeWriterBasebatch - the batch to write fromoffset - the row to start onlength - the number of rows to writeIOExceptionpublic void writeBatch(ColumnVector vector, int offset, int length) throws IOException
TreeWriterBasewriteBatch in interface TreeWriterwriteBatch in class TreeWriterBasevector - the vector to write fromoffset - the first value from the vector to writelength - the number of values from the vector to writeIOExceptionpublic void createRowIndexEntry()
throws IOException
TreeWriterBasecreateRowIndexEntry in interface TreeWritercreateRowIndexEntry in class TreeWriterBaseIOExceptionpublic void writeStripe(int requiredIndexEntries)
throws IOException
TreeWriterwriteStripe in interface TreeWriterwriteStripe in class TreeWriterBaserequiredIndexEntries - the number of index entries that are
required. this is to check to make sure the
row index is well formed.IOExceptionpublic void addStripeStatistics(StripeStatistics[] stats) throws IOException
TreeWriteraddStripeStatistics in interface TreeWriteraddStripeStatistics in class TreeWriterBasestats - the statistics for the new stripe across the
encryption variantsIOExceptionpublic long estimateMemory()
TreeWriterBaseestimateMemory in interface TreeWriterestimateMemory in class TreeWriterBasepublic long getRawDataSize()
TreeWriterpublic void writeFileStatistics()
throws IOException
TreeWriterwriteFileStatistics in interface TreeWriterwriteFileStatistics in class TreeWriterBaseIOExceptionpublic void flushStreams()
throws IOException
TreeWriterflushStreams in interface TreeWriterflushStreams in class TreeWriterBaseIOExceptionpublic void getCurrentStatistics(ColumnStatistics[] output)
TreeWritergetCurrentStatistics in interface TreeWritergetCurrentStatistics in class TreeWriterBaseoutput - an array that is filled in with the resultspublic void prepareStripe(int stripeId)
TreeWriterprepareStripe in interface TreeWriterprepareStripe in class TreeWriterBasestripeId - the next stripe idCopyright © 2013–2023 The Apache Software Foundation. All rights reserved.