Upstage 集成

pip install -U langchain-upstage

import os

os.environ["UPSTAGE_API_KEY"] = "YOUR_API_KEY"

from langchain_upstage import UpstageDocumentParseLoader

file_path = "/PATH/TO/YOUR/FILE.pdf"
layzer = UpstageDocumentParseLoader(file_path, split="page")

# 为提高内存效率，建议使用 lazy_load 方法逐页加载文档。
docs = layzer.load()  # 或 layzer.lazy_load()

for doc in docs[:3]:
    print(doc)

page_content='SOLAR 10.7B: Scaling Large Language Models with Simple yet Effective\nDepth Up-Scaling Dahyun Kim* , Chanjun Park*1, Sanghoon Kim*+, Wonsung Lee*†, Wonho Song*\nYunsu Kim* , Hyeonwoo Kim* , Yungi Kim, Hyeonju Lee, Jihoo Kim\nChangbae Ahn, Seonghoon Yang, Sukyung Lee, Hyunbyung Park, Gyoungjin Gim\nMikyoung Cha, Hwalsuk Leet , Sunghun Kim+ Upstage AI, South Korea {kdahyun, chan jun · park, limerobot, wonsung · lee, hwalsuk lee, hunkim} @ upstage · ai Abstract We introduce SOLAR 10.7B, a large language\nmodel (LLM) with 10.7 billion parameters,\ndemonstrating superior performance in various\nnatural language processing (NLP) tasks. In-\nspired by recent efforts to efficiently up-scale\nLLMs, we present a method for scaling LLMs\ncalled depth up-scaling (DUS), which encom-\npasses depthwise scaling and continued pre-\ntraining. In contrast to other LLM up-scaling\nmethods that use mixture-of-experts, DUS does\nnot require complex changes to train and infer-\nence efficiently. We show experimentally that\nDUS is simple yet effective in scaling up high-\nperformance LLMs from small ones. Building\non the DUS model, we additionally present SO-\nLAR 10.7B-Instruct, a variant fine-tuned for\ninstruction-following capabilities, surpassing\nMixtral-8x7B-Instruct. SOLAR 10.7B is pub-\nlicly available under the Apache 2.0 license,\npromoting broad access and application in the\nLLM field 1 1 Introduction The field of natural language processing (NLP)\nhas been significantly transformed by the introduc-\ntion of large language models (LLMs), which have\nenhanced our understanding and interaction with\nhuman language (Zhao et al., 2023). These ad-\nvancements bring challenges such as the increased\nneed to train ever larger models (Rae et al., 2021;\nWang et al., 2023; Pan et al., 2023; Lian, 2023;\nYao et al., 2023; Gesmundo and Maile, 2023) OW-\ning to the performance scaling law (Kaplan et al.,\n2020; Hernandez et al., 2021; Anil et al., 2023;\nKaddour et al., 2023). To efficiently tackle the\nabove, recent works in scaling language models\nsuch as a mixture of experts (MoE) (Shazeer et al.,\n2017; Komatsuzaki et al., 2022) have been pro-\nposed. While those approaches are able to effi- ciently and effectively scale-up LLMs, they often\nrequire non-trivial changes to the training and infer-\nence framework (Gale et al., 2023), which hinders\nwidespread applicability. Effectively and efficiently\nscaling up LLMs whilst also retaining the simplic-\nity for ease of use is an important problem (Alberts\net al., 2023; Fraiwan and Khasawneh, 2023; Sallam\net al., 2023; Bahrini et al., 2023). Inspired by Komatsuzaki et al. (2022), we\npresent depth up-scaling (DUS), an effective and\nefficient method to up-scale LLMs whilst also re-\nmaining straightforward to use. DUS consists of\nscaling the number of layers in the base model and\ncontinually pretraining the scaled model. Unlike\n(Komatsuzaki et al., 2022), DUS does not scale\nthe model using MoE and rather use a depthwise\nscaling method analogous to Tan and Le (2019)\nwhich is adapted for the LLM architecture. Thus,\nthere are no additional modules or dynamism as\nwith MoE, making DUS immediately compatible\nwith easy-to-use LLM frameworks such as Hug-\ngingFace (Wolf et al., 2019) with no changes to\nthe training or inference framework for maximal\nefficiency. Furthermore, DUS is applicable to all\ntransformer architectures, opening up new gate-\nways to effectively and efficiently scale-up LLMs\nin a simple manner. Using DUS, we release SO-\nLAR 10.7B, an LLM with 10.7 billion parameters,\nthat outperforms existing models like Llama 2 (Tou-\nvron et al., 2023) and Mistral 7B (Jiang et al., 2023)\nin various benchmarks. We have also developed SOLAR 10.7B-Instruct,\na variant fine-tuned for tasks requiring strict adher-\nence to complex instructions. It significantly out-\nperforms the Mixtral-8x7B-Instruct model across\nvarious evaluation metrics, evidencing an advanced\nproficiency that exceeds the capabilities of even\nlarger models in terms of benchmark performance. * Equal Contribution 1 Corresponding Author\nhttps : / /huggingface.co/upstage/\nSOLAR-1 0 · 7B-v1 . 0 By releasing SOLAR 10.7B under the Apache\n2.0 license, we aim to promote collaboration and in-\nnovation in NLP. This open-source approach allows 2024\nApr\n4\n[cs.CL]\narxiv:2...117.7.13' metadata={'page': 1, 'type': 'text', 'split': 'page'}