如何在idea中用maven配置spark和scala

1准备工作

首先需要在你电脑上安装jdk和scala以及开发工具Intellij IDEA，本文中使用的是win7系统，环境配置如下：

jdk170_15

scala2104

scala官网下载地址：>

在IDE上的设置

首先 下载的IntelliJ IDEA的 x（社区版） 因为目前它是最好的Scala的IDE 然后选择File |设置|插件 选择可用的选项卡来安装Scala插件

创建Maven项目

在IntelliJ IDEA中 选择文件|新建项目 然后选择Maven的模块创建一个Maven项目 然后修改如下所示的pom xml 添加所有你需要的设置 建立Scala的汇编语言

<project xmlnsxmlns= xmlns:xsi= instance

         xsi:schemaLocation= v _ _ xsd >

    <modelVersion> </modelVersion>

    <groupId> foo</groupId>

    <artifactId>myapp</artifactId>

    <packaging>war</packaging>

    <version> SNAPSHOT</version>

    <name>myapp Java EE Webapp</name>

    <url>;/url>

    <properties>

        <spring version> RELEASE</spring version>

     </properties>

    <repositories>

        <repository>

            <id> </id>

            <name>Repository hosting the jee artifacts</name>

            <url>;/url>

         </repository>

        <repository>

            <id></id>

             <name>Scala tools Maven Repository</name>

            <url>;/url>

        </repository>

        <repository>

             <id>wpt release</id>

            <url> repo/releases</url>

        </repository>

        <repository>

             <id>wpt snapshot</id>

            <url> repo/snapshots</url>

        </repository>

    </repositories>

    <pluginRepositories>

        <pluginRepository>

            <id></id>

            <name>Scala tools Maven Repository</name>

            <url>;/url>

         </pluginRepository>

    </pluginRepositories>

    <dependencyManagement>

        <dependencies>

            <dependency>

                 <groupId> springframework</groupId>

                <artifactId>spring context</artifactId>

                <version>${spring version}</version>

             </dependency>

             <dependency>

                <groupId> springframework</groupId>

                <artifactId>spring core</artifactId>

                <version>${spring version}</version>

            </dependency>

            <dependency>

                <groupId> springframework</groupId>

                 <artifactId>spring web</artifactId>

                <version>${spring version}</version>

             </dependency>

             <dependency>

                <groupId> springframework</groupId>

                 <artifactId>spring orm</artifactId>

                <version>${spring version}</version>

            </dependency>

        </dependencies>

    </dependencyManagement>

    <dependencies>

        <dependency>

            <groupId> h database</groupId>

            <artifactId>h </artifactId>

            <version> </version>

        </dependency>

        <dependency>

             <groupId> hibernate</groupId>

            <artifactId>hibernate entitymanager</artifactId>

            <version> GA</version>

        </dependency>

        <dependency>

            <groupId> springframework</groupId>

            <artifactId>spring orm</artifactId>

        </dependency>

        <dependency>

            <groupId>dev</groupId>

            <artifactId>wpt core</artifactId>

            <version> SNAPSHOT</version>

            <scope>test</scope>

        </dependency>

        <dependency>

            <groupId>dev</groupId>

            <artifactId>wpt runtime spring</artifactId>

             <version> SNAPSHOT</version>

        </dependency>

        <dependency>

             <groupId>javax</groupId>

            <artifactId>javaee web api</artifactId>

            <version> </version>

            <scope>provided</scope>

        </dependency>

        <dependency>

            <groupId>junit</groupId>

            <artifactId>junit</artifactId>

            <version> </version>

            <scope>test</scope>

        </dependency>

        <dependency>

            <groupId> scala lang</groupId>

            <artifactId>scala library</artifactId>

            <version> RC </version>

        </dependency>

        <dependency>

            <groupId> apache wicket</groupId>

            <artifactId>wicket</artifactId>

            <version> </version>

        </dependency>

        <dependency>

            <groupId> slf j</groupId>

            <artifactId>slf j log j </artifactId>

            <version> </version>

        </dependency>

    </dependencies>

    <build>

        <pluginManagement>

            <plugins>

                 <plugin>

                    <groupId> apache maven plugins</groupId>

                    <artifactId>maven piler plugin</artifactId>

                     <version> </version>

                </plugin>

                <plugin>

                     <groupId> scala tools</groupId>

                    <artifactId>maven scala plugin</artifactId>

                    <version> </version>

                </plugin>

            </plugins>

        </pluginManagement>

        <plugins>

            <plugin>

                 <groupId> apache maven plugins</groupId>

                <artifactId>maven piler plugin</artifactId>

                <executions>

                    <execution>

                        <phase>pile</phase>

                        <goals>

                            <goal>pile</goal>

                        </goals>

                    </execution>

                </executions>

                <configuration>

                    <source> </source>

                    <target> </target>

                </configuration>

            </plugin>

            <plugin>

                <groupId> apache maven plugins</groupId>

                 <artifactId>maven war plugin</artifactId>

                <version> beta </version>

                <configuration>

                    <failOnMissingWebXml>false</failOnMissingWebXml>

                 </configuration>

            </plugin>

            <plugin>

                 <groupId> scala tools</groupId>

                <artifactId>maven scala plugin</artifactId>

                <executions>

                    <execution>

                         <id>scala pile first</id>

                        <phase>process resources</phase>

                         <goals>

                            <goal>add source</goal>

                            <goal>pile</goal>

                        </goals>

                    </execution>

                    <execution>

                         <id>scala test pile</id>

                        <phase>process test resources</phase>

                         <goals>

                            <goal>testCompile</goal>

                        </goals>

                    </execution>

                </executions>

            </plugin>

        </plugins>

        <finalName>myapp</finalName>

    </build>

</project>

web xml中设置

接下来 修改main/webapp/WEB INF/web xml 从而确立了Wicket的过滤器 打开Spring的JPA实体管理器和Spring监听器 进行Spring的初始化

<xml version= encoding= UTF >

<web app version= xmlns= xmlns:xsi= instance

         xsi:schemaLocation= app_ _ xsd >

     <filter>

         <filter name>f </filter name>

        <filter class> springframework orm jpa support OpenEntityManagerInViewFilter</filter class>

    </filter>

    <filter>

         <filter name>f </filter name>

        <filter class> apache wicket protocol WicketFilter</filter class>

         <init param>

            <param name>applicationClassName</param name>

            <param value> foo myapp MyApp</param value>

         </init param>

    </filter>

    <filter mapping>

        <filter name>f </filter name>

        <url pattern>/app/</url pattern>

    </filter mapping>

    <filter mapping>

        <filter name>f </filter name>

        <url pattern>/</url pattern>

    </filter mapping>

    <context param>

        <param name>contextConfigLocation</param name>

        <param value>classpath:/beans xml</param value>

    </context param>

    <listener>

        <listener class> sprntext ContextLoaderListener</listener class>

     </listener>

</web app>

设置Spring bean访问和调用的数据库

要定义这些Spring bean 创建main/resources的beans xml以及一下的内容

<xml version= encoding= UTF >

<beansxmln eansxmlns=

        xmlns:xsi= instance

       xmlns:context=

        xmlns:tx=

       xsi:schemaLocation=

        beans xsd

        

        context xsd

        

        tx xsd >

     <context:ponent scan base package= foo myapp />

    <bean id= emf class= springframework orm jpa LocalContainerEntityManagerFactoryBean >

        <property name= persistenceXmlLocation value= classpath:/META INF/my persistence xml />

     </bean>

    <bean id= transactionManager class= springframework orm jpa JpaTransactionManager >

        <property name= entityManagerFactory ref= emf />

     </bean>

    <tx:annotation driven />

</beans>

读取数据库配置中persistence xml文件 在这里 您将获得一个主目录中名为MyApp的H 数据库 因此 创建main/resources/META INF文件的内容如下

<persistencexmlnspersistencexmlns=

            xmlns:xsi= instance

            xsi:schemaLocation= _ _ xsd

             version= >

   <persistence unit name= myapp transaction type= RESOURCE_LOCAL >

        <provider> hibernate ejb HibernatePersistence</provider>

        <properties>

           <propertynamepropertyname= nnection driver_class value= h Driver />

           <property name= nnection url value= jdbc:h :tcp://localhost/~/myapp />

            <propertynamepropertyname= nnection username value= sa />

            <propertynamepropertyname= nnection password value= />

           <property name= hibernate dialect value= hibernate dialect H Dialect />

           <property name= hibernate hbm ddl auto value= create />

        </properties>

   </persistence unit>

/persistence>

创建Wicket的页面

在这里 您将创建一个Wicket页面显示从数据库加载一些内容 因此 创建主/Scala文件夹 右键单击它并选择新建|Scala类 命名为我的页面类和投入 foo myapp包中 如下所示

package foo myapp

import apacl _

import basic Label

import list {ListItem ListView}

import apache wicket spring injection annot SpringBean

import apache wicket model CompoundPropertyModel

class MyPage extends WebPage {

  @SpringBean

   var ps: ProductService = _

  val productListView = newListView[Product]( productListView ps getAll) {

     def populateItem(item: ListItem[Product]) = {

      item setModel(newCompoundPropertyModel[Product](item getDefaultModelObject))

      item add(new Label( name ))

      item add(new Label( price ))

    }

  }

  add(productListView)

}

请注意 在创建ProductService对象加载的文件之后使用 此外 请注意该字段被分配到一个下划线（_） Scala编译为不可初始化的 而是留在默认状态 在你将覆盖作为编译之前 执行Spring bean的页面构造 现在 创建在src/main/resources//foo/myapp的的l文件

<!DOCTYPE HTML PUBLIC //W C//DTD HTML Transitional//EN >

<>

    <table border= >

        <tr wicket:id= productListView >

            <td wicket:id= name ></td>

            <td wicket:id= price ></td>

        </tr>

    </table>

</>

创建ProductService

右键单击在in the src/main/scala文件夹中的 foo myapp包并选择New | Scala的类 然后选择创建ProductService

package foo myapp

import java util List

trait ProductService {

  def getAll: List[Product]

}

这是接口 对创造一个DefaultProductService的Scala在同一个包内进行执行 *** 作

package foo myapp

import javax persistence {PersistenceContext EntityManager}

import springframework stereotype {Service}

import springframework transaction annotation Transactional

import springframework beans factory annotation Autowired

@Service

@Transactional

class DefaultProductService extends ProductService {

  @Autowired

  var products: Products = _

  def getAll = products getAll

}

请注意 把它使用类型文件命名为DAO

创建文件的DAO

要创建的DAO 创建一个名为在同一文件夹

package foo myapp

import java util List

trait Products {

   def getAll: List[Product]

}

然后创建在同一文件夹中的执行Scala的DefaultProducts文件

package foo myapp

import javax persistence {PersistenceContext EntityManager}

import springframework stereotype Repository

import java util List

@Repository

class DefaultProducts extends Products {

  @PersistenceContext

  var em: EntityManager = _

  def getAll = {

    em createQuery( select p from Product p ) getResultList asInstanceOf[List[Product]]

  }

}

创建entity程序

创建entity程序并将其映射到数据库

package foo myapp

import javax persistence {GeneratedValue Id Entity}

@Entity

class Product {

  @Id

  @GeneratedValue

  var id: Long = _

  var name: String = _

  var price: Double = _

}

创建Wicket的应用程序

最后 创建在同一个MyApp的Scala程序

package foo myapp

import apache wicket protocol WebApplication

dev wicketpagetest MockableSpringBeanInjector

class MyApp extends WebApplication {

  def getHomePage = classOf[MyPage]

   override def init = {

    MockableSpringBeanInjector installInjector(this)

  }

}

在这里 您指定为首页我的目录 植入Spring Bean中

运行应用程序

随着IntelliJ IDEA的版本不断改进 您将嵌入到应用服务器运行您的应用程序 这非常容易 在刚刚创建的测试 foo myapp的ManualTest在刚刚创建的测试 foo myapp包一类ManualTest/scala文件（不是main/scala文件） 以下内容是

package foo myapp

dev wicketpagetest {WebAppJettyConfiguration WicketAppJettyLauncher}

object ManualTest {

  def main(args: Array[String]) {

    val l = new WicketAppJettyLauncher

    l startAppInJetty(new WebAppJettyConfiguration)

  }

}

要运行的应用程序 首先需要运行数据库服务器上的H 程序 所以 在下载和解压 然后变成h /bin和运行h bat（或在Linux上h sh） ManualTest是在IDEA的编辑器上 在IDEA选择运行DEA 它将运行ManualTest Jetty的运行您的应用程序

为了测试它 请尝试通过在浏览//localhost: /app

不知道各位感觉如何 感谢各位支持！谢谢！

lishixinzhi/Article/program/Java/hx/201311/26175

在eclipse中，依次选择“File” –>“New” –> “Other…” –> “Scala Wizard” –> “Scala Project”，创建一个Scala工程，并命名为“SparkScala”。

右击“SaprkScala”工程，选择“Properties”，在d出的框中，按照下图所示，依次选择“Java Build Path” –>“Libraties” –>“Add External JARs…”，导入文章“Apache Spark：将Spark部署到Hadoop 220上”中给出的

assembly/target/scala-293/目录下的spark-assembly-081-incubating- hadoop220jar，这个jar包也可以自己编译spark生成，放在spark目录下的assembly/target/scala- 293/目录中。

切换到scala环境中存放testscala文件的文件夹下

先编译：

scalac testscala

再执行：

scala -classpath HelloWorld (这里HelloWorld为代码中的对象名称)

scala中执行外部命令(scalasysprocess)

发表回复

目前 scalasysprocess 已经封装的足够简单。参考： >

以上就是关于如何在idea中用maven配置spark和scala全部的内容，包括:如何在idea中用maven配置spark和scala、高级编程:Scala编程语言简介、用Java在各种框架下编译Scala项目等相关内容解答，如果想了解更多相关内容，可以关注我们，你们的支持是我们更新的动力！